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Preface 



With increased support for educational standards, teachers need resources to help them engage in stan- 
dards-based reform. The International Technology Education Association (ITEA) is publishing a series 
of addenda for this purpose. Measuring Progress: A Guide to Assessing Students for Technological 
Literacy offers teachers an approach to standards-based student assessment of technological literacy. 

ITEA originally developed Measuring Progress in 2002. Dr. Leonard Sterry wrote the first version of 
the document. It was field tested in ITEA’s Center to Advance the Teaching of Technology and Science 
(CATTS) Consortium states around the country. In March 2003, ITEA’s Technology for All Americans 
Project (TfAAP) released Advancing Excellence in Technological Literacy: Student Assessment, 
Professional Development, and Program Standards ( AETL ) (ITEA, 2003). In the spring of 2003, ITEA 
made the decision to have the TfAAP staff revise Measuring Progress to align it more fully with the 
new student assessment standards in AETL. Lisa Delany re-wrote the document in the summer of 2003, 
working in close consultation with Dr. Sterry to maintain the original intent and contextual framework 
of the document. Several other individuals helped make this document possible, and acknowledgements 
are provided in Appendix A. 

Measuring Progress is a resource for teachers to use as they plan and implement standards-based student 
assessment. Section 1 is an overview of how standards relate to assessing student technological literacy. 
Section 2 provides an approach to standards-based student assessment. Section 3 describes several 
assessment tools and methods. Section 4 details assessment principles that teachers should consider 
when designing student assessment. And Section 5 suggests some applications for assessment data as 
well as evaluation strategies for ensuring effective student assessment. There are also several resources in 
the appendices, including forms that can be photocopied for teachers to use as they work toward 
standards-based reform of student assessment in their own laboratory-classrooms. 

Measuring Progress is most useful when users are already familiar with the technology content stan- 
dards in Standards for Technological Literacy: Content for the Study of Technology (STL) (ITEA, 
2000/2002) and the companion standards for student assessment, professional development, and pro- 
gram enhancement in AETL. However, teachers may use Measuring Progress as a bridge to understand- 
ing the vision of the standards as it pertains to student assessment. ITEA is developing additional 
addenda that examine topics such as standards-based programs, professional development, and 
curricula. ITEA welcomes feedback on all of the guides in this addenda series as we work together 
to encourage technological literacy for all students. 
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Advancing Technological Literacy: ITEA Professional Series 



The Advancing Technological Literacy: ITEA Professional Series is a set of publications developed by 
the International Technology Education Association (ITEA) based on Standards for Technological 
Literacy (ITEA, 2000/2002) and Advancing Excellence in Technological Literacy (ITEA, 2003). The 
publications in this series are designed to assist educators in developing contemporary, standards-based 
K-12 technology education programs. This exclusive series features: 

• Direct alignment with technological literacy standards, benchmarks, and guidelines. 

• Connections with other school subjects. 

• Contemporary methods and student activities. 

• Guidance for developing exemplary programs that foster technological literacy. 

Titles in the series include: 

Technological Literacy Standards Series 

• Standards for Technological Literacy: Content for the Study of Technology’ 

• Advancing Excellence in Technological Literacy: Student Assessment, Professional 
Development, and Program Standards 

• Technology for All Americans: A Rationale and Structure for the Study of Technology 

Addenda to Technological Literacy Standards Series 

• Measuring Progress: A Guide to Assessing Students for Technological Literacy 

• Realizing Excellence: A Guide for Exemplary Programs in Technological Literacy 

• Planning Learning: A Guide to Developing Technology > Curricula 

• Teaching Technology: A Guide for Professional Development 

Standards-Based Technological Study Series 

Elementary School Level Resources 

• Models for Introducing Technology: A Standards-Based Guide 

• Technology > Starters: A Standards-Based Guide 

Middle School Level Resources 

• Exploring Technology: A Standards-Based Middle School Model Course Guide 

• Invention and Innovation: A Standards-Based Middle School Model Course Guide 

• Technological Systems: A Standards-Based Middle School Model Course Guide 

• Teaching Technology: Middle School, Strategies for Standards-Based Instruction 
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High School Level Resources 

• Foundations of Technology’: A Standards-Based High School Model Course Guide 

• Engineering Design: A Standards-Based High School Model Course Guide 

• Impacts of Technology: A Standards-Based High School Model Course Guide 

• Technological Issues: A Standards-Based High School Model Course Guide 

• Teaching Technology: High School, Strategies for Standards-Based Instruction 

Standards-Based Technological Study Units 

Elementary School Level Resources 

• Invention, Innovation, and Inquiry (I 3 ) Units 

— Invention: The Invention Crusade 
— Innovation: Inches, Feet, and Hands 
— Communication: Communicating School Spirit 
— Manufacturing: The Fudgeville Crisis 
— Transportation: Across the United States 
— Construction: Beaming Support 
— Power and Energy >: The Whispers of Willing Wind 
— Design: Toying with Technology 
— Inquiry: The Ultimate School Bag 
— Technological Systems: Creating Mechanical Toys 

• Kids Inventing Technology Series (KITS) 

Secondary School Level Resources 

• Humans Innovating Technology Series (HITS) 
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SECTION 1 



Standards-Based 
Student Assessment of 
Technological Literacy 




This section provides an overview of standards-based assessment of 

student technological literacy. 








SECTION 1 



The study of 
technology is 
distinct and 
different from 
educational 
(instructional) 
technology, which 
utilizes tools, such 
as computers, 
audiovisual 
equipment, and 
mass media, to 
enhance teaching 
and learning in all 
school subjects. 




Technology is the 
innovation, 
change, or 
modification of 
the natural 
environment to 
satisfy perceived 
human needs and 
wants. 



Facilitating Technological Literacy 

We live in a technological world. Living in the twenty-first century requires 
much more from every individual than a basic ability to read, write, and per- 
form simple mathematics. . . . Citizens of today must have a basic understand- 
ing of how technology affects their world and how they exist both within and 
around technology .... Students need and deserve the opportunity to attain 
technological literacy through the educational process. (ITEA, 2003, pp. 1-2) 

Society is becoming increasingly dependent upon technology and technological 
advancements. We, as educators, need to prepare students with the knowledge 
and abilities needed to interact with the technological world. The promise of the 
future lies not in technology alone, but in people’s ability to use, manage, assess, 
and understand it (ITEA, 1996; 2003). Students need to study technology to 
develop technological literacy. 



Technological 
literacy is the 
ability to use, 
manage, assess, 
and understand 
technology. 



Educational standards offer a focused approach to education. The International 
Technology Education Association (ITEA) developed four nationally-recognized 
sets of educational standards on content, student assessment, professional devel- 
opment, and programs for the study of technology. These standards provide the 
basis for strong technology programs and can move the study of technology 
forward in a united national effort. 



Educational Standards for Technological Literacy 

Educational standards are statements about what is valued that can be used for 
making a judgment of quality (ITEA, 2000/2002). Every teacher of technology 
should be familiar with the standards for technology content, student assessment, 
professional development, and programs. 
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Standards for Technological Literacy: Content for the Study 
of Technology (STL) (ITEA, 2000/2002) provides the con- 
tent basis for the study of technology. STL standards (see 
Appendix B) specify what every student should know and 
be able to do in order to attain technological literacy. The 
standards include the application of both knowledge and 
abilities to real-world situations. 

However, content is not enough to ensure the effective 
study of technology. So, ITEA developed Advancing 
Excellence in Technological Literacy: Student 
Assessment, Professional Development, and Program 
Standards (AETL) (ITEA, 2003). AETL is based on STL. 
Thus, AETL provides the means for implementing STL 
in the laboratory-classroom. AETL contains three 
separate but related sets of standards: 

1 . The student assessment standards (see Appendix C) identify criteria to 
ensure effective assessment of student technological literacy. 

2. The professional development standards (see Appendix D) ensure effec- 
tive and continuous in-service and pre-service education for teachers of 
technology. 

3. The program standards (see Appendix E) detail criteria for technology 
program enhancement. 

Measuring Progress will focus on the student assessment standards ( see Table 1). 
However, in the big picture, student assessment is only one program component. 





Table 1. Student Assessment Standards from AETL 

A- 1 . Assessment of student learning will be consistent with 

Standards for Technological Literacy: Content for the Study of 
Technology (STL). 

A-2. Assessment of student learning will be explicitly matched to 
the intended purpose. 

A-3. Assessment of student learning will be systematic and derived 
from research-based assessment principles. 

A-4. Assessment of student learning will reflect practical contexts 
consistent with the nature of technology. 

A-5. Assessment of student learning will incorporate data collection 
for accountability, professional development, and program 
enhancement. 



Student 

assessment refers 
to the process of 
collecting 
evidence on 
student learning, 
understanding, 
and abilities to 
inform instruction 
and provide 
feedback to the 
learner. 



Measuring 
Progress 
examines student 
assessment as it 
applies to 
individual 
laboratory- 
classrooms. While 
ITEA recognizes 
the need to use 
large-scale 
assessment 
instruments to 
determine state 
and/or national 
student 
technological 
literacy. 
Measuring 
Progress does not 
discuss large- 
scale assessment. 
Large-scale 
assessment 
involves 
examining the 
learning of a large 
number of 
students, such as 
across a 
state/province/ 
region or nation. 
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SECTION 1 



Content standards 
refers to the 
standards in STL. 
Technological 
literacy standards 
refers to the 
standards in both 
STL and AETL. 



Connecting Student Assessment With Standards: 
The Big Picture 

Planning and delivering standards-based student assessment requires that we 
pause, take a step back, and consider the big picture of the technology pro- 
gram in which we are teaching. AETL defines program as everything that 
affects student learning. This includes content, professional development, 
curricula, instruction, student assessment, and the learning environment, 
implemented across grade levels (see Figure 1). 

Measuring Progress is intended to help teachers align student assessment with 
technological literacy standards. Additional ITEA addenda examine such top- 
ics as standards-based programs, professional development, and curricula. 



Figure 1. Selected Program Components 




Learning 

Environment 



Student 

Learning 



Curricula 



Instruction 



Professional 

Development 



Content 



Technology 

Program 



Planning Standards-Based Student Assessment 

Student assessment is the systematic, multi-step process of collecting evidence 
on student learning, understanding, and abilities and using that information to 
inform instruction and provide feedback to the learner, thereby enhancing learn- 
ing (ITEA, 2003). Appendix F provides suggestions for teachers, administra- 
tors, and policymakers to consider as they answer the following fundamental 
questions of standards-based planning: 
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Where are we 
now? The 
answer to this 
question will 
depend upon 
how we respond 
to two interme- 
diate questions: 

What is the cur- 
rent level of stu- 
dent technologi- 
cal literacy in 
our laboratory- 
classrooms' ? and 
What is the cur- 
rent state of 
student assess- 
ment ? We need 
to examine how 

student assessment facilitates technological literacy in our classrooms, 
schools, and school districts. “If teachers are reasonably sure about what 
their students currently know, then teachers can more accurately tailor . . . 
[student assessment according] to what students need to know” (Popham, 
1999, p. 2). “Where are we now” depends upon whether student assessment 
is standards-based and whether student assessment actually measures 
technological literacy. 

Where do we want to go? We must identify the outcomes we want from 
teaching and learning — what should graduates of our schools know and be 
able to do related to technology ? The standards to which a school or school 
district is committed can answer this question and may include national, 
state, and/or local standards. STL and AETL are the only nationally- accept- 
ed sets of educational standards for technological literacy. STL “does not 
prescribe an assessment process for determining how well students are 
meeting the standards, although it does provide criteria for such an assess- 
ment” (ITEA, 2000/2002, p. 13). Measuring Progress asserts that student 
assessment should be aligned with technological literacy standards. 

How are we going to get there? Linking student assessment practices with 
standards requires more than showing that specific standards are being 
assessed. We have to consider the strategies that we will use to advance 
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SECTION 1 



Evaluation refers 
to the process of 
collecting and 
processing 
information and 
data to determine 
how well a design 
meets the 
requirements and 
to provide 
direction for 
improvements. 



student technological literacy. Planning standards-based student assessment 
requires that we consider how we can best support students as they seek to 
attain technological literacy (Carr & Harris, 2001). The answer is a standards- 
based approach to teaching, learning, and student assessment. Measuring 
Progress is primarily intended to help teachers “get there.” 

What knowledge and abilities must educators possess to get there? 

Professional development, both pre-service and in-service, helps teachers “get 
there” by providing opportunities based on teachers’ needs and, ultimately, the 
needs of their students. Appendix D is a listing of the professional development 
standards from AETL. Teachers and other educators should seek multiple oppor- 
tunities to engage in comprehensive and sustained personal professional growth. 
In other words, teachers of technology need to remain current with the changing 
nature of technology and research in education. 

How will we know when we have arrived? We will know that we have arrived 
when student assessment data show that students have reached the desired level 
of technological literacy As we compare the evidence from student assessment 
with our teaching and learning expectations, we can make judgments about the 
level of understanding students have reached. If students do not perform well, 
we know that we need to evaluate the technology program and make revisions. 
Both successes and failures should be reported to program stakeholders. The 
goal is for all students to achieve technological literacy. Section 5 suggests 
some applications for assessment data as well as evaluation strategies. 

A Word of Caution: Standards-Based vs. 
Standards-Reflected 

Standards-feaser/ assessment starts with content standards as a base, then develops 
specific learning goals, and then establishes assessment criteria against which 
understanding is judged. The real question becomes: What do students know, do, 
and ultimately, understand! By using a standards-based approach, student learning 
is assessed against criteria based on standards instead of program inputs, such as 
existing curricula and instruction. In other words, the teacher avoids the “check- 
list” trap of saying, “Oh, I’m already doing this,” “I just talked about this yester- 
day,” or “I covered this in the course introduction.” It is important to note that for 
assessment to be standards-based, the curricula and the instruction must also be 
based upon the assessment criteria. 

Standards -reflected assessment starts with assessment tools and methods and 
attempts to make connections to the standards. In other words, standards become an 
afterthought and are not the basis for developing assessment practices. Standards- 
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reflected assessment is misleading, because it gives the impression that practices are 
standards-based when they are not. Revising existing assessment tools and methods 
to show connections to standards does not necessarily ensure that technological 
literacy is the intended outcome of assessment. Standards-reflected assessment 
“. . . makes an explicit commitment to standards, but this ... is not enough for plan- 
ning standards-based learning. Too often the connections are weak or insufficient, 
consequently some standards get left out or the overall picture of standards imple- 
mentation remains unclear” (Carr & Harris, 2001, pp. 18-19), such as in the “check- 
list” trap mentioned in the preceding paragraph. Table 2 provides a comparison of 
standards-based student assessment with standards-reflected student assessment. 



Table 2. Comparison of Standards-Based Assessment with 
Standards-Reflected Assessment 



Standards-Based Student 
Assessment 

1 . Identify STL standards and 
benchmarks to serve as basis for 
the content to be assessed. 

2. Consider AETL standards and 
guidelines. 

3. Define assessment criteria — 
“what a student should look 
like.” 

4. Identify an assessment tool or 
method that will deliver content 
in a manner consistent with 
AETL. 

5. Develop lessons or activities to 
deliver STL content. 

6. Gather evidence of student learn- 
ing, using the selected assess- 
ment tool or method. 

Result: Student assessment that 

measures technological literacy 

consistent with STL & AETL. 



Standards-Reflected Student 
Assessment 

1 . Start with a lesson or activity. 

2. Identify the content being 
delivered by the lesson or activity. 

3. Identify STL standards and 
benchmarks that might align with 
the lesson or activity content. 

4. Select an assessment tool or 
method. 

5. Consider how the selected tool 
or method addresses AETL 
standards. 



Result: Student assessment that 
measures technological literacy when 
a "match" can be made between the 
lesson content and STL standards. 



Sharing the Vision 

Aligning student assessment with standards may seem overwhelming at first. But 
with a shared vision for developing technologically literate students, everyone — 
students, teachers, parents, administrators, and communities — can work together to 
help all students attain technological literacy through K-12 education. 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 



7 



SECTION 1 








SECTION 2 

An Approach to 
Standards-Based 
Student Assessment 




This section provides an approach to standards-based student 
assessment with step-by-step instructions. The approach is applied 
to the course and unit levels of teaching and learning. A similar 
approach may be used at the program level. 



SECTION 2 



Imagine a laboratory-classroom where student assessment not only measures techno- 
logical literacy but also advances it. Student assessment provides students with oppor- 
tunities to solve practical, real-world problems relevant to the concerns of society. 
Student assessment incorporates multiple standards from Standards for Technological 
Literacy: Content for the Study of Technolog}’ {STL) (ITEA, 2000/2002) and high- 
lights the interrelationships among technologies and the connections between technol- 
ogy and other disciplines. Student assessment engages students in hands-on, minds-on 
activities that foster critical thinking, decision making, and problem solving related to 
the use, management, and evaluation of the designed world. In such a classroom or 
laboratory, student assessment occurs as an integral part of teaching and learning, and 
assessment results improve the teaching and learning process for all students. 

Both STL and Advancing Excellence in Technological Literacy: Student 
Assessment, Professional Development, and Program Standards ( AETL ) (ITEA, 
2003) assert that the study of technology must occur in a comprehensive, articu- 
lated fashion from kindergarten through Grade 12. Students learn basic techno- 
logical concepts at the early elementary level and build upon knowledge and abil- 
ities at the middle and high school levels. Student assessment provides evidence 
that learning is facilitated across grade levels and disciplines by occurring “... in 
conjunction with the ongoing nature of the study of technology throughout 
Grades K-12” (ITEA, 2003, p. 21). 



We should not 
view student 
assessment as an 
event — a single 
moment-in-time 
test — but as a 
varied collection 
of evidence over 
time (Wiggins & 
McTighe, 1998). 



Evidence refers to 
the information 
collected that 
demonstrates or 
proves student 
understanding. 



Holistic Assessment: A Collection of Evidence 

We should not view student assessment as an event — a single moment-in-time 
test — but as a varied collection of evidence over time (Wiggins & McTighe, 
1998). Therefore, we must view student assessment as a process. It involves quan- 
tifying, describing, observing, reporting, and giving feedback. It needs to be prac- 
tical and manageable. Most importantly, the process should assist students in 
attaining technological literacy by being standards-based. 

Figure 2 illustrates a step-by-step (or multi-step) approach to a standards-based 
student assessment process. The approach does not look at any single assessment 
instance to determine student understanding. Instead, it requires educators to look 
at a collection of assessment results to arrive at a holistic judgment of student 
understanding. While the approach presents five steps, experienced teachers know 
that organizing teaching and learning does not always occur in a linear fashion. 

As teachers become comfortable planning standards-based student assessment, 
they will revisit and re-examine steps to help thoroughly link student assessment 
to technological literacy standards. 
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It is important to note that the steps at the course level and unit level are identical 
in the approach presented in Figure 2. A course lasts for a specified period of 
time (e.g., semester, year) and is designed around a specific school subject. 
Courses are composed of units. A unit is an organized series of learning activi- 
ties, lectures, projects, and other teaching strategies that focuses on a specific 
topic related to the course curriculum (ITEA, 2003). 



Figure 2. An Approach to Standards-Based 
Assessment of Technological Literacy 



COURSE LEVEL 

Step 1. Identify content standards, then select benchmarks. 
Step 2. Extract and organize content. 

Step 3. Define assessment criteria. 



UNIT LEVEL 



Unit #3* 




Unit #2 




Unit #1 

Step 1. Identify content standards, then select 
benchmarks. 

Step 2. Extract and organize content. 

Step 3. Define assessment criteria. 

Step 4. Select and use assessment tools/methods. 
Step 5. Make use of assessment results 



*Note: The number of units in a course will depend upon how 
content is extracted and organized in Step 2 at the course level. 



Step 4. Select and use assessment tools/methods. 
Step 5. Make use of assessment results. 



The steps that follow will lead you through the approach in Figure 2. It is 
extremely important to identify whether you are planning at the course level or 
unit level. When working at the unit level, Steps 1-3 will be based upon your 
responses to Steps 1-3 at the course level. Please photocopy Appendix H (for 
course level planning) or Appendix I (for unit level planning) and fill it in as you 
complete Steps 1-5, which follow. If you do not have a copy of STL, you may 
view the content standards online at http://www.iteawww.org. 



The standards- 
based assessment 
approach 
incorporates the 
backward design 
concept (Wiggins 
fr McTighe, 1998). 
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SECTION 2 




The term content standards here and elsewhere in Measuring Progress consistent- 
ly refers to the standards in STL. The text that follows will also indicate alignment 
with the student assessment standards in AETL. Additionally, a vignette is embed- 
ded within the text to provide a “snapshot” of how a teacher might apply the stan- 
dards-based approach to student assessment in his or her laboratory-classroom. 
Please see page 14 for an example. 



The goal is to 
meet all of the 
standards through 
the benchmarks. 
ITEA does not 
recommend that 
teachers eliminate 
any of the 
benchmarks over 
the K-12 
experience; 
however, 
teachers may find 
it desirable to add 
additional 
benchmarks. 



Step 1 : Identify Content Standards, Then Select 
Appropriate Benchmarks 

Begin with a clear picture of what you want by identifying the content standards 
that will guide student assessment in your laboratory-classroom. The standards in 
STL define what students should know, be able to do, and ultimately understand 
about technology in a very broad sense. Standards provide direction and a spirit 
of intent, not a complete list of important concepts. It is not necessary to address 
all 20 standards in any given unit, course, or grade. However, it is necessary to 
address all of the STL standards and benchmarks within a grade “band” (K-2, 
3-5, 6-8, or 9-12). 



It is vital to begin by looking at the standards, rather than the benchmarks that fol- 
low the standards, to capture the breadth of content in STL. The content standards 
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that are selected should integrate in a way that will enable you to deliver variety 
in technological content (AETL Student Assessment Standard 4, Guideline B ). 

Prior to applying this approach to an individual course or unit, you will 
need to work with teachers across grade levels and disciplines to ensure 
that each of the standards is addressed at increasing levels of complexity 
each time the content is encountered. In addition to identifying which tech- 
nology content standards will be taught and assessed, you will benefit by docu- 
menting other content area standards that will be addressed (AETL Student 
Assessment Standard 1, Guidelines A & B). You may find it helpful to use the 
Responsibility Matrix Form provided as Appendix G. Please see the vignette on 
page 14 for an example. 

Once you have identified the standards you will assess in your course or 
unit, record them appropriately on your Standards-Based Student 
Assessment Form. If you have completed a Responsibility Matrix Form, you 
will need to reference it to address those standards for which you and your stu- 
dents will be held accountable. Obviously, if you are working at the course level, 
you will select more standards and benchmarks than you will at the unit level. It 
is generally recommended that you assess no more than two or three stan- 
dards in each unit. 

Once you have selected content standards, you will then choose the bench- 
marks that “uncover” concepts necessary to develop an understanding of those 
standards. Benchmarks in STL are provided in grade bands (K-2, 3-5, 6-8, 
and 9-12). Benchmarks should add detail to the concepts that will be taught 
and assessed. As with the content standards, students do not need to encounter 
every benchmark at each grade level within a grade band, within each course, 
or within each unit. Please record the benchmarks that you will address on 
your Standards-Based Student Assessment Form. The vignette on page 15 
provides an example. 



Nationally 
developed content 
standards in other 
academic areas 
include (but are 
not limited to): 

• National 
Science 
Education 
Standards (NRC, 
1996) 

• Benchmarks for 
Science Literacy 
(AAAS, 1993) 

• Principles and 
Standards for 
School 
Mathematics 
(NCTM, 2000) 

• Geography for 
Life: National 
Geography 
Standards 
(GESP, 1994) 

• National 
Standards for 
History (H CHS, 
1996) 

• Standards for 
the English 
Language Arts 
(NCTE, 1996) 

• National 
Educational 
Technology 
Standards for 
Students: 
Connecting 
Curriculum and 
Technology 
(ISTE, 2000) 
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Vignette 



Snapshot of the Approach in Action 

Note: The vignette is intended to provide a snapshot of the approach taken to align student 
assessment with technological literacy standards. It provides examples. It does not provide a 
comprehensive examination of each step. 

Ms. Toledano, a 6 th grade technology teacher, is interested in aligning student assessment in her 
laboratory-classroom with technological literacy standards. She begins by working with other teachers to 
identify the grade levels at which individual standards will be taught and assessed. Using the 
Responsibility Matrix Form (Appendix G in Measuring Progress), she develops a matrix to document 
teacher responsibilities for addressing the content standards in STL as well as related content standards 
in other disciplines (see Figure 3). 



APPENDIX G 

Responsibility Matrix Form 



Directions: Page 1 of this form should be used to indicate which standards in Standards for Technological Literacy 
(STL) will be addressed at each grade level of the technology program. Fill in this form using “X” to indicate maxi- 
mum coverage, “V” to indicate moderate coverage, and “O” to indicate minimal coverage. 



Responsibility Matrix Form Page 1 





STL Coverage in the Technology Program 


STL Standards 


Elementary Level Classrooms 


Technology Laboratory-Classrooms 




K 


1 / 


3 


4 


5 


6 


7 


l 10 


11 


12 


STL 1. Students will develop an understanding of the 
characteristics and scope of technology. 


X 


X 


1 x 


V 


0 


X 


v 


) x 


V 


V 


STL 2. Students will develop an understanding of the core concepts 
of technology. 


V 


V 


l 0 


0 


X 


V 


X ] 


) X 


0 


V 


STL 3. Students will develop an understanding of the relationships 
among technologies and the connections between technology and 
other fields of study. 


V 


V 


0 


X 


X 


0 


X 1 


I v 


V 


X 


STL 4. Students will develop an understanding of the cultural, 
social, economic, and political effects of technology. 


0 


V 


1 0 


X 


V 


0 


V 


) 0 


0 


V 


STL 5. Students will develop an understanding of the effects of 
technology on the environment. 


0 


o ( 


V 


0 


X 


0 


O 1 


X 


0 


0 



Figure 3 



14 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 



Vignette, Continued 



In planning her course, Ms. Toledano reviews her Responsibility Matrix Form and consults STL to iden- 
tify the content standards she will address. She looks to the benchmarks for further detail. Ms. Toledano 
records the standards and benchmarks she intends to teach and assess in her course on the Course Level 
Standards-Based Student Assessment Form (see Figure 4). 



APPENDIX H 

Course Level 

Standards-Based Student Assessment Form 

Step 1. SELECT CONTENT STANDARDS, THEN IDENTIFY BENCHMARKS 

Identify and document the content standards that will serve as the basis for student assessment. Then select 
the benchmarks that add detail to the concepts that will be taught and assessed. Multiple copies of this page 
may be needed. 

STOP! And confirm: Review the selected content standards and benchmarks to verify that they include 
variety in technological content. 

Step 2. EXTRACT AND ORGANIZE CONTENT 

What are the specific understandings students should possess having engaged in the content of the standards and 
benchmarks? What are the big ideas students should learn to ultimately understand the standards and benchmarks? 



Content Standard 


Benchmark 


STL 6. Students will develop an 
understanding of the 
role of society in the 
development and use of 
technology. 


Benchmark. D. 

Throughout history, new technologies have resulted from the demands, values, and 
interests of individuals, businesses, industries, and societies. 


Big Idea: 


Big Idea: 


Big Idea: 



Figure 4 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 



15 




SECTION 2 



Step 2: Extract and Organize Content 



As detailed as the benchmarks are, they may need additional interpretation. From 
the benchmarks, you must identify and document the enduring concepts or “big 
ideas” that students should learn so they will ultimately understand the standards 
and benchmarks in STL. “The term enduring [concept] refers to the big ideas, the 
important understandings, that we want students to ‘get inside of’ and retain after 
they’ve forgotten many of the details” (Wiggins & McTighe, 1998, p. 10). 

In some instances, the enduring concepts may be the benchmarks themselves; in 
other instances, the enduring concepts may be derived from elements of the 
benchmark. For example, STL Standard 2, Benchmark N states: “Systems think- 
ing involves considering how every part relates to others.” This benchmark could 
serve as the big idea to be learned. By contrast, STL Standard 17, Benchmark L 
states: “Information and communication technologies include the inputs, process- 
es, and outputs associated with sending and receiving information.” In this 
instance, the big ideas might include individual examination of inputs, processes, 
and outputs. 

You might find it helpful to answer the question: What are the specific under- 
standings students should possess having engaged in the content of the standards 
and benchmarks ? We must provide students with opportunities for critical think- 
ing and decision making (AETL Student Assessment Standard 4, Guideline C). 
We need to be able to gather evidence of student learning within the cognitive, 
psychomotor, and affective learning domains (AETL Student Assessment 
Standard 1, Guidelines C, D, & E). Additionally, we should provide students with 
opportunities to engage in technological problem solving — design, engineering 
design, troubleshooting, research and development, invention and innovation, and 
experimentation (AETL Student Assessment Standard 4, Guideline A). 

Record the big ideas that you will address on your Standards-Based Student 
Assessment Form. Please see the vignette on page 17 for an example. 
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Vignette, Continued 



After completing Steps 1-3 of her course level planning, Ms. Toledano decides that the selected standards 
could be addressed in a unit on forensics. She has already completed Steps 1 and 2 by choosing appropriate 
standards and benchmarks from her course level planning form. To complete Step 3, Ms. Toledano extracts 
the important understandings from the standards and benchmarks she selected. She records these ideas on the 
Unit Level Standards-Based Student Assessment Form (see Figure 5). 



APPENDIX I 

Unit Level 

Standards-Based Student Assessment Form 

Step 1. SELECT CONTENT STANDARDS, THEN IDENTIFY BENCHMARKS 

Identify and document the content standards that will serve as the basis for student assessment by 
re-examining Steps 1 and 2 at the course Level. Then seLect the benchmarks that add detail to the 
concepts that will be taught and assessed. 

STOP! And confirm: Review the seLected content standards and benchmarks to verify that they include 
variety in technological content. 

Step 2. EXTRACT AND ORGANIZE CONTENT 

What are the specific understandings students should possess having engaged in the content of the standards and 
benchmarks? What are the big ideas students should learn to ultimately understand the standards and benchmarks? 



Content Standard 


Benchmark 


STL 6. Students will develop an 
understanding of the role 
of society in the 
development and use 
of technology. 


Benchmark. D. 

Throughout history, new technologies have resulted from the demands, values, and interests of 
individuals, businesses, industries, and societies. 


Big Idea: 

DNA technologies available 
for use today provide greater 
efpciencg and accuracy than 
previously used technologies. 


Big Idea: 

Economic, political, and 
cultural issues influence the 
selection and use of 
technologies for forensic 
investigation. 


Big Idea: 



Figure 5 
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Step 3: Define Assessment Criteria 



Evidence refers to 
the information 
that demonstrates 
or proves a level 
of understanding. 
Teachers collect 
evidence of 
student learning 
and judge that 
evidence against 
assessment 
criteria to identify 
the attained level 
of technological 
literacy in relation 
to the content 
standards. 



Assessment criteria are indicators that suggest the level of understanding attained 
by students. Assessment criteria provide the basis for teaching and learning by 
capturing the essential ingredients of the content being measured. They are writ- 
ten to provide cues to you and your students about what significantly indicates 
student technological literacy. Curricula, instruction, assessment tools and 
methods, and the learning environment are developed after assessment crite- 
ria have been established and are based on the assessment criteria. You will 
use the assessment criteria to judge student understanding — a collection of evi- 
dence over time — to determine the level of student technological literacy. 

As assessment criteria are established, we must consider that student learning will 
be influenced by a variety of factors including student commonality and diversity 
— interests, cultures, abilities, socio-economic backgrounds, and special needs. 

To allow for student commonality and diversity and the reality that all students 
will not attain a consistent level of understanding all of the time, we must delin- 
eate assessment criteria at varying levels to accurately assess student progress 
toward technological literacy. 



To write the assessment criteria, begin by examining the big ideas. Ask yourself, 
what should my students look like having learned the big ideas ? This might be 
considered the acceptable or “at target” level of understanding. Then specify crite- 
ria for learning that exceed your expectations, or are “above target.” And finally 
define criteria that do not meet your expectations or are below your expectations, 
which might be considered “below target.” Thus, the assessment criteria will be 
written for at least three levels of understanding. The Standards-Based Student 
Assessment Forms provide matrices for writing assessment criteria differentiated 
at three, four, and five levels. Choose which matrix you will use, and record the 
assessment criteria on your Standards-Based Student Assessment Form. 
Alternatively, you may choose to develop and use a matrix of your own. Please 
refer to the vignette on page 1 9 for an example. 
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Vignette, Continued 



Ms. Toledano is ready to complete Step 3 for her unit on forensics. Examining “Big Idea: DNA tech- 
nologies available for use today provide greater efficiency and accuracy than previously used technolo- 
gies,” she establishes and records assessment criteria on her Unit Level Standards-Based Student 
Assessment Form (see Figure 6). She categorizes the levels of understanding as “above target,” “at tar- 
get,” and “below target.” 



Note: The form below provides only one example of assessment criteria appropriate for Ms. Toledano s 
unit. A fully-developed form would likely contain more assessment criteria. 



Step 3: DEFINE ASSESSMENT CRITERIA 

What are the expectations for student Learning? Define criteria at Levels which meet, exceed, and falL below 
your expectations. 

NOTE: Defining assessment criteria at various levels requires that you first determine the number of levels that 
will be assigned. You may choose to use any of the three tables that follow or develop one of your own. 



Big Ideas 


Assessment Criteria 


Below Target 


At Target 


Above Target 


DNA technologies available for use 
today provide greater efficiency and 
accuracy than previously used 
technologies. 


identification of DNA 
technologies currently in 
use for forensic 
investigation. 


identification of DNA 
technologies currently in 
use for forensic 
investigation compared 
ivith those available 15 
years ago. 


identifi cation of DNA 
technologies currently in 
use for forensic 
investigation compared 
u/ith those available 15 
gears ago. Evidence of 
research of DNA 
technologies to be 
available in the future. 



























Figure 6 
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Step 4. Select and Use Assessment Tools and/or 
Methods 



You will need to use a variety of assessment tools and methods to accurately 
determine student understanding. You will be able to collectively examine the evi- 
dence from those individual tools and methods in light of the assessment criteria 
you developed in Step 3 to judge overall student technological literacy. Selection 
of tools and methods should be based on the content being assessed and the type 
of evidence being gathered. There are many individual assessment tools and 
methods available, and Section 3 of Measuring Progress describes several of 
them. Also, there are several general principles that are vital to the application of 
any individual assessment practice. These are explained in Section 4. 

Identify and record the tools and methods that you will use to gather 
evidence of student understanding on your Standards-Based Student 
Assessment Form. 



Comparison with 
the backward 
design process 
outlined by Grant 
Wiggins and Jay 
McTighe (1998) 
may reveal a 
seeming 

discrepancy in the 
order of Steps 3 and 
4. However, there is 
no discrepancy, as 
the Wiggins and 
McTighe process is 
applied on a smaller 
scale, at the level of 
designing individual 
assessment 
instances, whereas 
the approach put 
forth in Measuring 
Progress enables 
those individual 
assessment 
instances to be 
based upon the 
overall assessment 
criteria for the 
course or unit. 




Step 5. Make Use of Assessment Results 

After Steps 1-4 have been applied through the teaching and learning process, you 
will be able to make judgments about what your students learned by comparing 
the evidence collected to the assessment criteria. You may find it helpful to con- 
sider the questions: Are my students technologically literate? Did my students 
learn what I intended them to learn? Why or why not? The assessment criteria 
define aspects of student technological literacy. By examining the evidence as a 
whole — rather than considering the individual instances of evidence gathering — 
you will be better able to determine overall student technological literacy. 

It is important to note that this process is not the same as the process of assigning 
grades. If the students do not “measure up” to the assessment criteria, there may 
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be reasons that have little to do with student ability or effort, such as a need to 
adjust curricula or instruction. While assigning grades is a necessary part of the 
teaching process, Measuring Progress does not focus on grading per se. 
Measuring Progress asserts that student assessment should be used to advance 
student technological literacy, not simply as a process forjudging it. 

Document how you will use student assessment results for your course or 
unit on your Standards-Based Student Assessment Form. You may choose to 
do this before delivering the course, or you may do this after you see the results 
of student assessment — or preferably both. 

Assessment is an empty process unless the results are used to make positive 
change. Teachers assess students to improve teaching and learning; however, the 
results may be used in a variety of ways: 

• Improving teaching and learning. 

• Assigning grades. 

• Monitoring progress. 

• Identifying levels of technological literacy. 

• Determining instructional effectiveness. 

• Communicating results. 

• Marketing and promotion. 

• Guiding professional development decisions. 

• Guiding program enhancement decisions. 

Section 5 details some other applications for assessment data as well as sugges- 
tions for evaluating the approach used to assess students (see Appendix J). 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 



21 



SECTION 2 




Mil \ 



SECTION 3 

Student Assessment 
Tools and Methods 




This section describes several assessment tools and methods. 
Student technological literacy may be assessed using additional tools 
and methods that are not mentioned here. In any case, the tool or 
method selected must show or provide evidence of student learning. 
Note: All individual assessment tools and methods must adhere to 
the assessment principles presented in Section 4. 



SECTION 3 



"While the data 
produced by 
student 

assessment are 
used by many 
people for a 
variety of 
purposes, the 
primary purpose 
of assessment 
should be to 
improve teaching 
and learning." 
(ITEA, 2003, p. 18) 



Individual 
instances of 
evidence 
gathering (i.e., 
multiple-choice 
tests, student 
portfolios, etc.) 
will be examined 
separate and 
apart from the 
holistic 

examination of the 
collective 
evidence gathered 
throughout 
teaching and 
learning. Section 3 
focuses primarily 
on the individual 
instances of 
evidence 
gathering, 
whereas Section 
2 describes a 
more holistic 
approach to 
student 
assessment. 



Traditionally, the need to assign grades has been a major reason for assessing stu- 
dents. The National Research Council’s (NRC) report, Knowing What Students 
Know: The Science and Design of Educational Assessment (2001b) indicates that 
the three broad purposes of assessment include 1) assisting learning, 2) measuring 
individual achievement, and 3) evaluating programs. “While the data produced by 
student assessment are used by many people for a variety of purposes, the pri- 
mary purpose of assessment should be to improve teaching and learning” 
(ITEA, 2003, p. 18), which corresponds with NRC Purposes 1 and 2 above. 

Measuring Progress asserts that we should use student assessment to monitor stu- 
dent progress toward technological literacy and to enhance teaching and learning. 

Gathering Evidence 

We use assessment tools and methods to collect evidence of student learning. 
Different tools and methods are used in different situations. The use of any sin- 
gle assessment practice is not adequate. Students have different levels of intelli- 
gence, learning styles, and capabilities. They demonstrate achievement in differ- 
ent ways. Therefore, students should be assessed often using a variety of tools and 
methods. We compare the collective evidence gathered by individual student 
assessment tools and methods with the assessment criteria (see Step 3 in Section 
2). In this way, we can determine student technological literacy over time, for 
example, over the duration of a unit or course. 

As we review the evidence gathered by the tool or method and compare it to the 
assessment criteria, we must judge the evidence based upon established parame- 
ters that reflect our expectations for learning. Scoring devices, such as rubrics, are 
useful to establish these parameters. The use of scoring devices is further dis- 
cussed at the end of this section. 

Selection of individual assessment tools and methods is based on the content, the 
purpose, and the audience for assessment results. Therefore, before any specific 
tool or method can be designed, we must identify the purpose and the audi- 
ence of the assessment tool or method. 

Purpose of Assessment 

The assessment purpose should be driven by the evidence of student understand- 
ing to be gathered as well as the intended use of student assessment results. 

Before designing an individual tool or method, we should write a purpose state- 
ment to clarify our intentions and ensure that the tool or method selected is 
appropriate to the type of evidence needed. We might use a multiple-choice test to 
assess student knowledge of design terminology, whereas we would more likely 
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select a model or prototype to assess student abilities to apply the design process. 
We must also make the assessment purpose and learning expectations clear to our 
students (AETL Student Assessment Standard 2, Guideline A). 

Additionally, the purpose of the tool or method should be driven by how we 
intend to use the results of the assessment practice. If we clarify in advance that 
the purpose of the assessment instance is to, for example, quickly check that our 
students are “getting it” before moving the lesson forward, we would choose a 
formative assessment tool or method over one that is summative in nature. Like- 
wise, if our purpose is to assess the level of student technological literacy at a 
given point in time, we would be more likely to select some summative measure. 

Audience for Assessment Results 

Assessment results may be used to inform individuals and groups of progress 
educators are making toward developing a technologically literate populace. The 
audience for a specific assessment tool or method should be clearly stated and 
considered in the process of development (AETL Student Assessment Standard 2, 
Guideline B). Please note that audience refers to those who will see the assess- 
ment data, not those who will use the assessment tool or method. Table 3 lists 
various audiences for assessment results and gives examples of the information 
needed by those audiences. 
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Table 3. Audiences for Assessment Results 


Audience 


Information Need(s) 


Students 


How close am I coming to my potential ability? 

How am I doing in school (relative to the study of technology)? 
Will I meet graduation requirements? 

Am I prepared to obtain my career goal? 


Parents 


How well is my child doing in school? 

Are there areas where I can help her/him attain technological literacy? 


Teachers 


How are my students doing? 

What adjustments do I need to make in my instruction to ensure that my students attain 
technological literacy? 

Are all of the content standards and benchmarks in STL being addressed? 

Do classroom practices reflect all of the standards and guidelines in AETL1 


Department Chairs 


What support must I provide to the teachers to help them enhance student learning? 
Are all of the content standards and benchmarks in STL being addressed? 

Do classroom practices reflect all of the standards and guidelines in AETL1 


Principals 


How are students in my building meeting technological literacy standards? 

Where have we shown improvements in technological literacy learning? 

Where should I focus resources to improve technological literacy learning in this school? 


Curriculum 

Coordinators 


What gaps remain in our district's curriculum with respect to technological literacy standards? 
Which instructional programs are most effective? 

What are the professional development needs of district staff? 


Local District 
Superintendents 


How well are the schools in my district assisting students in attaining technological literacy? 
Which programs need to be revised? 

What resources are needed to improve student technological literacy learning across the district? 


Local School Boards 


Is our district using its resources to advance student technological literacy? 
What improvements are needed in our schools? 


Taxpayers 


How well are the schools in my community preparing students to function as responsible, 
technologically literate adults? 


Business & Industry 


To what extent are students technologically literate and prepared for the world of work? 
Will our workforce continue to be competitive in a global economy? 


State Superintendents 


What programs can the state's educational system provide to support technological literacy 
learning in our schools? 


State Boards of 
Education 


Is the state making the best use of its resources to support student technological literacy learning? 


Colleges & 
Universities 


How well prepared are students to continue their formal education with respect to the study of 
technology? 


Educational 

Researchers 


What factors influence the study of technology in our schools? 

What programs are needed to improve technological study in our schools? 


State Policymakers 

(Legislators & Governors) 


What evidence is available to show the effectiveness of statewide educational policy with respect 
to technological literacy? 

What do state level indicators tell us about our state's educational policies as they relate to 
technological literacy? 


National Leaders 


How do states compare to each other in the development of technological literacy? 

How do our students compare to those in other countries? 

What nationwide educational programs are needed to strengthen student technological literacy? 


Society 


ALL OF THE ABOVE 



(Levande, 2001, September, adapted with permission) 



Selected Assessment Tools and Methods 

The assessment tools and methods that follow are presented alphabetically and are summarized by: 

• Description - Brief overview of the tool/method. 

• Guidelines - Some suggestions to consider as the tool/method is developed. 

•Advantages - Strengths to consider in selecting the tool/method. 

• Limitations - Precautions to consider in selecting the tool/method. 
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Concept Mapping 

Description. A concept map, also referred to as a “web” or “graphic organizer,” is a representation 
that students create to detail relationships among ideas (see Figure 7). An event or issue is written at 
the center of the concept map. Students brainstorm answers to the question, “if this, then what?” 
They are instructed to show relationships among the ideas by connecting the concepts with lines. 



Figure 7. Sample Concept Map 




Guidelines 

• Present students with a general concept. 

• Ask students to brainstorm as many ideas as 
they can that relate to the concept. Students 
might begin by answering the question: What 
do I know about the concept ? 

• Flave students arrange their brainstormed 
ideas into groups, so that the elements of a 
group represent similar ideas. Groups of 
words should be labeled with terms that 
define all of the elements of a particular 
group. Some or all of the brainstormed ideas 
may not fit into any group. Students will write 
these ideas into circles on the graphic. 

• Flave students connect similar ideas (groups) 
with lines to show a relationship. Every idea 
should be connected to another idea in some 
fashion. 

• Concept mapping is a creative process without 
a distinctive beginning, middle, or end. 
Individual students will approach it differently. 



• Allows students to consider the “big” picture 
by making connections between new informa- 
tion and prior knowledge. 

• Enables students to illustrate how they have 
“made sense” of the concept. 

• Can provide direction for future learning 
activities based on student understanding 
and/or misunderstanding. 

• Relatively quick way to determine “where” 
students are and assess progress. 

Limitations 

• Concepts and/or ideas presented may be 
vague. 

• Students may not identify as many concepts as 
necessary for getting a good look inside their 
ways of thinking about a concept. 
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Debate 

Description. A debate is an open discussion “for” or “against” an issue or question. This issue or 
question is often controversial so that opposing views can be presented. The viewpoints presented, 
however, should be based on researched evidence. Debates are usually conducted between two teams 
of three or four students, who perform in front of a classroom audience. 



Guidelines 

Prior to the debate: 

• Select a debate topic that is current, relevant, 
appropriate to students, and researchable with 
existing resources. 

• Provide students with the debate “logistics” — 
purpose, rules and procedures, and time allot- 
ted for conducting research. 

During the debate: 

• A moderator states the problem to both teams. 

• Five minutes are provided to each team for 
formal presentations (affirmative, then 
negative). 

• Five minutes are provided to each team for 
rebuttals (affirmative, then negative). 

• The moderator requests questions and/or con- 
tributions from the audience (affirmative, then 
negative; repeat as needed). 

• Three-minute summary speeches are given by 
each team (affirmative, then negative). 

• The moderator opens the floor to questions. 

The debate team members and the audience are 
provided equal opportunities for participation. 

• The moderator summarizes any new informa- 
tion presented. 



Advantages 

• Supports student learning of controversial topics. 

• Allows students to draw their own conclu- 
sions. 

Limitations 

• “In-depth” learning of the topic only by the 
students engaged in research. 

• Objective assessment may be difficult. 



(Adapted from Henak, 1988, p. 150) 
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Demonstration / Presentation 



Description. A demonstration is an active presentation. It generally enables students to show how 
something is done or how a particular thing or concept operates. A presentation is more formal than 
a demonstration. It is a structured communication with limited interaction. A presentation allows 
students to tell about the concept or process. Visual aids, such as audiovisual media, are often used 
in both demonstrations and presentations. 



Guidelines 

• Provide students with a list of expectations for 
their demonstrations/presentations. 

• Inform students of the importance of prepara- 
tion — spending sufficient time practicing, 
having all needed materials and tools available 
and in place before the demonstration/presen- 
tation, etc. 

• Encourage each student to use visual aids so 
that all students in the audience can see the 
demonstration/presentation. 

• Provide feedback to students during the 
demonstration/presentation by asking 
questions. 



Advantages 

• Enables students to demonstrate their abilities 
or inform others of their understanding. 

• Can replicate “real world” situations. 

• Can be used to assess a range of topics. 

Limitation 

• Requires substantial amount of time in com- 
parison to some other tools and methods. 



(Adapted from Hill, 1988, pp. 140-141) 
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Design Brief 

Description. A design brief is a written plan that engages students in the design problem-solving 
process. Design briefs generally present students with a situation and ask them to solve a life prob- 
lem. Students use the design process to make decisions and produce plans to convert resources into 
products or systems that meet human needs and wants or solve problems. 



Guidelines 

• Identify an appropriate context for the design 
problem. The context should describe a life 
situation that connects to the students. 

• Define the challenge by describing the 
problem to be solved. 

• Identify the criteria and constraints, or require- 
ments and limitations. Criteria are desired 
specifications of a product or system. 
Constraints are limitations to the design 
process, and may include appearance, funding, 
space, materials, and human capabilities 
(ITEA, 2000/2002). 

• Specify the resources that students may use as 
they generate a solution to the problem. 

• Indicate criteria that students should use to 
evaluate their design to determine whether or 
not the proposed solution solves the problem 
and meets the requirements of the design 
brief. 



Advantages 

• Students solve a problem in an organized and 
analytical manner. 

• Provides in-depth insight into student under- 
standing and learning. 

• Causes students to consider various criteria 
and constraints. 

Limitation 

• Defining a meaningful problem can be 
difficult. 



(Adapted from Wright & Brown, 2004) 
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Discussion / Interview 



Description. Discussion provides an opportunity for teachers to interact with students and to listen 
to students as they interact with each other. Teachers can query students, coaching them, causing 
them to realize that situations should be viewed from multiple perspectives. Teachers can find out 
what students know and what misconceptions students possess. Students are required to express 
their viewpoints, not just their knowledge. Teachers can make judgments about student understand- 
ing as they listen to students “make sense” of topics, issues, or content. Teachers can use open-ended 
questioning to initiate discussion (see p. 36). 

While discussion is relatively informal, an interview is more structured. It includes a planned 
sequence of questions, similar to a job interview. Teachers collect data on student knowledge and 
abilities at a certain point in time by soliciting student responses to a series of questions. Interviews 
may be conducted with individual students or groups of students. 

Guidelines 

• Ensure that the classroom atmosphere is invit- 
ing to student discussion. Students should feel 
free to share or not to share their ideas, view- 
points, and opinions. 

• Identify probing questions in advance to guide 
the discussion or interview. This will prevent 
the conversation from straying. 

• Ask specific questions. Do not accept absolute 
responses such as “yes” or “no,” or generali- 
ties such as, “it was too hard” or “it was fun.” 

Require students to offer explanations that 
validate their opinions. 

• Encourage participation by many students. 

Discourage a few students from dominating 
the discussion. 

• Engage students in discussion regularly, 
throughout an activity, lesson, unit, etc. 

• Record feedback. 

• Utilize feedback to determine necessary 
instructional adjustments. 



Advantages 

• Offers potential to engage the entire class in 
discussion. 

• Provides an opportunity for students to criti- 
cally think about the content. 

• Students can learn from each other. 

• Students learn on their own by vocalizing their 
understandings and justifying their arguments. 

Limitations 

• Requires a substantial amount of time in com- 
parison to other, more direct approaches. 

• Potential to stray from the subject matter. 

• Some students may participate more often 
than other students. 



(Adapted from Jones, Bagford, & Wallen, 1979) 
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Journal / Log 

Description. A journal or a log is a collection of written, periodic (daily/weekly) entries that docu- 
ment learning. Students may use journals or logs as a record of understanding, reflection, and/or 
opinion. Students may choose to include student summaries, drawings, or reports. Teachers may 
require students to reflect on and respond to a given prompt, such as, how would your life be affect- 
ed if transportation systems became obsolete, or simply allow students to document their own 
thoughts or understandings. A journal may be part of a portfolio (see p. 37). 



Guidelines 

• Encourage students to dedicate a notebook to 
serve as their journal. 

• Prompt students to write in their journals. 
Devote time regularly for student journaling. 

• Consistently monitor student journaling. 

While it is not necessary to review student 
entries in detail, it is advisable to ensure that 
students periodically record their understand- 
ings, reflections, or opinions. 

• Allow students to be open in their documenta- 
tions. Some students only write what they 
think teachers want to see. Students should be 
told that information in the joumal/log is for 
learning and not necessarily scoring. 

• Inform students that their entries will be con- 
fidential and available to the teacher and stu- 
dent or student only. 



Advantages 

• Allows teachers to see understandings and 
misunderstandings of students in the learning 
process. 

• Provides a record of daily activity or work 
accomplished. 

• Allows students to reflect on their own past 
learning experiences. 

• Documents accounts of individual student 
learning. Provides students a means for self 
reflection. 



Limitations 

• Students may write what they think teachers 
are looking for. This distorts what teachers 
think students are “getting” or “not getting.” 

• Students may tend to get off track and not 
fully explain their understandings, questions, 
or opinions. 
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Modeling / Prototyping 

Description. Modeling is the process by which students develop a detailed, small-scale visual, math- 
ematical, or three-dimensional representation of an object or design (model). Prototyping is the 
process by which students develop a full-scale working model of the final solution (prototype) 
(ITEA, 2000/2002). Modeling and prototyping are processes that require students to demonstrate 
their knowledge and abilities to meet specified criteria and/or constraints. Teachers gauge student 
understanding by considering student abilities to satisfy pre-determined parameters. 



Guidelines 

• Select content to ensure that students demon- 
strate knowledge of technological concepts 
and not simply create a model or prototype for 
the sake of doing. 

• Select content that is relevant to students and 
their experiences outside of the classroom. 

• Consider the process in which students will 
engage. 

• Provide students with clear expectations so 
they will know what will be assessed: func- 
tion, aesthetics, process, construction tech- 
niques, safety considerations, etc. 

• Provide students with opportunities to work 
together collectively rather than individually. 



Advantages 

• Allows students to demonstrate understanding 
by applying their knowledge and abilities. 

• Students are able to learn about technology by 
“doing” technology. 

• Provides practical opportunities to connect to 
the real world. 



Limitations 

• Requires a substantial amount of time in com- 
parison to some other tools and methods, both 
in the creation of the model or prototype and 
the critique of it. 

• Critiquing the model or prototype involves 
subjectivity. 
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Multiple-Choice Test 

Description. A multiple-choice test poses a series of questions to students and requires that they 
select the correct answer, or the “best” answer, from a series of choices. Multiple-choice tests are 
among the more commonly used paper-and-pencil tests. 



Guidelines 

• Begin with a question that you want answered. 
State it as a direct question, or make it an 
incomplete statement. 

• State the problem in the stem. The stem 
should provide as much needed content as 
possible so students do not have to “work” at 
making sense of the test item. The stem should 
make sense all by itself. 

• Provide three to five “alternatives” or possible 
responses, each having some merit, with only 
one being the correct answer. 

• Include the correct alternative as well as incor- 
rect alternatives, or distracters, as possible 
answer options. 

• Use distracters that make the student think 
about each choice carefully. Answer choices 
should be comparable — avoid giving cues by 
making the length of the answers similar and 
maintaining consistency in the complexity of 
responses. Eliminate the use of “all of the 
above” or “none of the above.” 

• CAPITALIZE, bold face, and/or underline 
negative wording, including instances in which 
the answer is an exception. 

• Use grammatically correct stems and choices, 
specifically when the choices need an “a” or 
“an.” 

• Place correct answers in random order 
throughout the test. 

• Avoid giving away answers to questions or 
statements in other test items. 



Advantages 

• Less costly than other forms of testing. 

• Easy to administer to a large number of 
students. 

• Can sample a wide variety of learning targets. 

• “Fast” tool to test student knowledge of 
specific content. 

• Can be used diagnostically to improve 
instruction. 



Limitations 

• Can fail to assess higher-order thinking skills. 

• Provides just one measure of student learning. 

• Are constructed upon the assumption that 
knowledge can be represented by an accumu- 
lation of bits of information and that there is 
only one “right” answer. 

• Encourages teaching to the test, which 
narrows the curriculum. 



(Adapted from Georgia Department of 
Education, 2000; Popham, 1999) 
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Observation 



Description. Observation refers to the act of absorbing information by seeing and/or listening to stu- 
dents. Teachers witness student understanding by watching and/or listening to students as they 
demonstrate their knowledge and abilities. Teachers may observe students as they attempt to solve a 
design problem, or observe students as they discuss an issue such as global warming. 



Advantages 

• Provides opportunity to assess students while 
they are engaged in an activity. 

• Judgments of student learning can be made 
initially, in-progress, rather than after-the-fact. 

• Recorded evidence can later be evaluated for 
accuracy. Decisions regarding instructional 
adjustments can be made. 

Limitations 

• Requires substantial amount of time in com- 
parison to some other tools or methods. 

• Tendency to watch for “bad” behaviors. 
Attention may focus on what students are 
doing wrong rather than on what they are 
doing correctly. 

(Adapted from Hart, 1 994) 



Figure 8. Sample Observation Form 


Observed Behaviors 


What was Observed 


Notes, Comments 


Student groups are on task. 






Students are actively discussing the 
issue. 




Students are using prior concepts to 
solve the problem. 




Students are exhibiting a positive 
attitude. 




Students are demonstrating an 
understanding of ... [technological 
problem solving]. 




Students are using multiple strategies to 
address the problem. 




Students are using a variety of skills to 
present a group solution. 




Others 





(Enger & Yager, 2001, p. 98) 



Guidelines 

• Specify focus for observations. Determine 
what you are “looking” for. 

• Record observations in writing. Figure 8 illus- 
trates a sample observation form that could be 
used when observing students. 

• Note the typical as well as the atypical. 
Observations of the routine are just as valu- 
able as observations of the extraordinary. 

• Repeat observations. One instance does not 
make a pattern. 

• Synthesize evidence from different contexts to 
increase the validity of observations. 

• Observe all students, often and regularly. 
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Open-Ended Questioning 

Description. “An open-ended question is a question that does not have just a single right answer but 
can be answered in a number of ways” (Carin & Bass, 2001, p. 144). Teachers present a question, 
situation, or scenario, and students respond, verbally, in writing, or by creating something. Open- 
ended questioning may be used to generate a discussion (see p. 31). Teachers assess higher-order 
thinking skills and are able to “get at” understandings that are otherwise difficult to measure. 
Freedman ( 1 994) suggests asking questions that engage students in: 

1. Analysis: Students are challenged to analyze a situation and to suggest possible solutions, requir- 
ing that they think critically about the situation, gather information pertaining to the situation, and 
write, illustrate, model, or communicate possible solutions. 

2. Comparison: Students are challenged to examine a situation and compare selected elements of the 
situation, requiring that they critically examine the strengths and weaknesses and/or pros and cons 
of the situation. 

3. Description: Students are required to respond to the situation by offering a description. The 
description provides detail of the situation and might be of such things as an event or a process. 

4. Evaluation: Students are challenged to gather information, analyze it, and make judgments. 

5. Fiction: Students write stories based on information, patterns, and trends from data gathered and 
analyzed that may not be based upon present-day realities. 

6. Problem solving: Students respond to a question by detailing the process in which they are 
engaged. Students may detail the gathering, analysis, and interpretation of data to identify 
resulting trends and forecast possible outcomes. 



Guidelines 

• Identify probing questions in advance to 
ensure that the conversation does not stray 
from the subject matter. 

• Be specific in the questions that are asked. Do 
not accept absolute responses such as “yes” or 
“no,” or generalities such as, “it was too hard” 
or “it was fun.” Require students to offer 
explanations that validate their opinions. 

• Engage students in discussion regularly, 
throughout an activity, lesson, unit, etc. 

• Record feedback. 

• Utilize feedback to determine modifications 
needed to the current lesson as well as the 
focus for future lessons. 



Advantages 

• Students are challenged to consider content 
and/or ideas from different perspectives. 

• Students can learn from each other. 

• Students learn on their own by vocalizing their 
understandings and justifying their arguments. 

• Provides an opportunity for students to engage 
in in-depth thinking about the content. 

Limitations 

• Requires some time to develop probing 
questions. 

• Discussions easily stray from the subject matter. 
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Portfolio 

Description. A portfolio is a formal or informal, systematic, and organized collection of student 
work that includes results of research, successful and less successful ideas, notes on procedures, and 
data collected. Portfolios take various forms, from photographs depicting student growth and under- 
standing to specialized electronic journals showing work completed over a period of time (ITEA, 
2003). A review of work presented in a portfolio provides an impression of how well a student is 
doing. (Appendix K contains an example of a scoring rubric for use with a portfolio.) 



Guidelines 

• Determine the kind of evidence students 
should include in their portfolios — for exam- 
ple, tell students whether they are showcasing 
their understanding or documenting their 
progress. 

• Clearly identify and communicate the focus of 
the portfolio. Students must be able to deter- 
mine the type of evidence to include in their 
portfolios (for example, objects, drawings, 
plans, written statements, and photos). 

• Insist that students organize their portfolios. 
This enables efficient review of the portfolios. 



Advantages 

• Provides a means for collecting a variety of 
evidence of student learning. 

• Allows students to make decisions about 
selecting representations of their “best” work. 

• Allows students to demonstrate their progress 
toward technological literacy rather than 
requiring an absolute level of performance at a 
given instance. 

• Provides an opportunity to link the evidence 
collected directly to instruction. 

• Provides an opportunity for students to con- 
nect what they are doing in school with their 
interests outside of school. 

• Provides opportunities for teachers to review 
student portfolios as a means of reflecting on 
the effectiveness of their teaching. 

• Provides a holistic picture of student learning. 



Limitations 

• Requires a substantial amount of time in com- 
parison to some other tools and methods. 

• Critiquing the portfolio involves subjectivity. 



(Adapted from Barton & Collins, 1997) 
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Project 

Description. A project involves students in practical contexts reflective of real world experiences, 
requiring that they demonstrate their understanding by proposing solutions to issues of relevance 
and value to others. “Open-ended in nature, a project poses multiple solutions and engages students 
in a ‘whole’ situation, one that encourages discovery of its parts, relationships, meaning, and 
resolution” (Campbell, Campbell, & Dickinson, 1996, p. 253). 



Guidelines 

• Identify important concepts or practices and 
determine an open-ended project that encom- 
passes such knowledge. Identify practical and 
relevant concepts with which students have 
some connection and about which they can be 
excited. 

• Involve students in planning the various 
aspects of a project. At times, students should 
also determine their own projects. 

• Guide students through various stages of proj- 
ect initiation, implementation, refinement, 
presentation, reflection, assessment, and 
revision. 

• Select student drafts and final work to submit 
for documentation during and upon comple- 
tion of the project. 

• Ask students to think back over their learning 
processes and personal growth achieved as a 
result of the project. 

• Have students present their projects to an 
audience of classmates, parents, community 
members, or others who will support as well 
as offer constructive criticism of student 
efforts. 



• Assess the project from numerous 
perspectives. 

• When students have completed their projects, 
reflect on what their work reveals about 
them — interests, strengths, challenges, 
whether they are independent or collaborative 
workers, and what interests emerged that 
might be addressed in future projects. 

Advantages 

• Provides opportunities for students to transfer 
classroom learning to practical contexts. 

• Provides opportunities for interdisciplinary 
learning enabling students to see the relation- 
ships among technologies and the connections 
between technology and other fields of study. 

Limitation 

• Requires substantial amount of time in com- 
parison to some other tools and methods. 



(Adapted from Campbell et al., 1996) 
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Self Assessment / Peer Assessment 

Description. Self assessment encourages students to reflect upon and evaluate their own learning. 
Student reflection should occur periodically so students can keep informed of their own progress 
toward technological literacy. Peer assessment encourages students to provide feedback to one 
another related to the learning that has occurred. Students may be asked to peer assess a partner, a 
group of students, or the entire class. 



Guidelines 


Advantages 


• Provide students with a clear understanding of 
what is expected of them and what is to be 
learned. 


• Students gain a better understanding and 
appreciation for the assessment process by 
participating in it. 


• Consider allowing students to help define 
parameters for their assessment. This will 
allow them to understand the learning goal 
and thus be able to self/peer assess on a more 
“grounded” level. 


• Students are able to ascertain what knowledge 
and abilities they have gained during the 
learning process. 

• Students can learn from their own mistakes. 


• Allow students to self correct. 


Limitations 

• If students do not have a clear understanding 
of what is expected of them, then the self/peer 
assessment will not be successful. 

• Students may not take the self-assessment or 
peer-assessment process seriously. Giving one- 
self or one’s friend an “A” for the sheer pur- 
pose of giving it can be a downfall if the pur- 
pose of the self/peer assessment is focused on 
scoring purposes alone. 
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True-False Tests 

Description. True-false test items require that students respond to statements by selecting one of 
only two potential answer options — “true” or “false.” 



Guidelines 

• Eliminate extraneous wording and use of dis- 
tracters. 

• Use uncomplicated vocabulary and concise 
sentence form. Simple is better! 

• Avoid use of negative statements. The use of 
negative statements should only be considered 
when they are the best way to elicit the correct 
answer. Emphasize negative words using ALL 
CAPITAL LETTERS, bold font, and/or 
underlining . 

• Give specific statements of opinion with the 
source. 

• Prepare an equivalent number of true state- 
ments as false statements. 



Advantages 

• Relatively easy to construct. 

• Requires only a short time to administer. 

Limitation 

• Students have a 50-50 chance of selecting cor- 
rect answers and can answer correctly even if 
they do not know the information. 



(Adapted from Georgia Department of 
Education, 2000) 
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Scoring Devices 



Scoring, grading, measuring, judging, or assigning value — whether we are using 
a multiple-choice test or a design brief, at some point we need to consider indi- 
vidual student achievement. Many of the assessment tools and methods described 
in this section provide subjective evidence of student learning rather than objec- 
tive evidence. In such instances, we must use scoring devices to help us make 
more objective decisions based on specific criteria. Checklists and rubrics are 
two scoring devices that we might consider. 

Checklist 

A checklist is an evaluative device, generally in the form of a simple listing of 
criteria that define expectations for student responses. Please see Figure 8 on 
page 35. 

Rubric 

A scoring rubric is essentially a two-dimensional matrix that provides criteria 
against which to assess student performance. One axis of the matrix presents a 
set of concepts, elements, traits, or big ideas, while the other axis defines levels 
of achievement. The two axes, one vertical and the other horizontal, form rows 
and columns. Figure 9 provides a template that teachers might find helpful to use 
when developing a scoring rubric. Appendix K contains some examples of com- 
pleted scoring rubrics. 
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Scoring rubrics help the teacher by: 



• Enabling the teacher to more objectively critique evidence of student 
learning. The gathered evidence determines how well students really under- 
stand the content. 

• Generating feedback for improved instruction. Knowing how well students 
truly understand important concepts helps the teacher modify instruction to 
improve learning. 

• Monitoring student progress. Teachers can make students aware of their 
strengths and areas where improvement is necessary. 

• Grading student performance. Sound grading is based on sound assessment 
tools and methods. Sound assessment practices are based on clear targets. 

Scoring rubrics help the student by: 

• Identifying what is expected. Students are aware of what they are supposed 
to know and how well they should know it. 

• Generating feedback for improved learning. Students can improve their 
learning by knowing where they need to improve and how they can improve 
their performance. 

• Establishing criteria for grading that are based on clear expectations and 
objective evidence. Student scoring is consistent with the expectations about 
which they were informed. Students are aware of what is expected and know 
how their performance will be scored in accordance with these expectations. 

Considerations for developing rubrics include: 

• Rubrics should not be overly complex. That is, they should contain six or 
fewer big ideas for assessment and five or fewer levels of performance. 

• Rubrics can be used over and over again. It is up to the discretion of the 
teacher to tailor the rubric to ensure content accuracy and grade level appro- 
priateness. 

• Criteria within a rubric should be consistent. For example, if most of the cri- 
teria are calling for measures of quality in a performance, stay with quality 
throughout the rubric. Don’t suddenly switch to a quantity-measure along the 
way. Both quality and quantity are appropriate measures; they just need to be 
stated consistently in the criteria throughout the rubric. 

• Students can be involved in generating rubric criteria. This can be an inter- 
esting way for them to discuss and help determine what is important for 
them to learn. 
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• Acceptable performance should first be defined, then criteria can be written 
for performance that exceeds or falls short. The following questions may prove 
helpful to teachers as they differentiate levels of performance and establish 
criteria defining the characteristics that represent each level of performance. 

1 . What would be sufficient and revealing evidence of understanding? 

2. What performance tasks must anchor the unit and focus the instructional 
work? 

3. How will the teacher be able to distinguish between those students who 
really understand and those who do not (though they may seem to)? 

4. Against what criteria will the teacher distinguish work? 

5. What misunderstandings are likely? How are they checked for? 



Figure 9. Rubric Template 


(Describe the task or performance that this rubric is designed to assess here.) 




Beginning 


Developing 


Accomplished 


Exemplary 


Score 




1 


2 


3 


4 




Stated 
Objective or 
Big Idea 


Description of 
identifiable 
characteristics 
reflecting a beginning 
level of performance. 


Description of 
identifiable 
characteristics 
reflecting 
development and 
movement toward 
mastery of 
performance. 


Description of 
identifiable 
characteristics 
reflecting mastery 
of performance. 


Description of 
identifiable 
characteristics 
reflecting the 
highest level of 
performance. 




Stated 
Objective or 
Big Idea 


Description of 
identifiable 
characteristics 
reflecting a beginning 
level of performance. 


Description of 
identifiable 
characteristics 
reflecting 
development and 
movement toward 
mastery of 
performance. 


Description of 
identifiable 
characteristics 
reflecting mastery 
of performance. 


Description of 
identifiable 
characteristics 
reflecting the 
highest level of 
performance. 




Stated 
Objective or 
Big Idea 


Description of 
identifiable 
characteristics 
reflecting a beginning 
level of performance. 


Description of 
identifiable 
characteristics 
reflecting 
development and 
movement toward 
mastery of 
performance. 


Description of 
identifiable 
characteristics 
reflecting mastery 
of performance. 


Description of 
identifiable 
characteristics 
reflecting the 
highest level of 
performance. 




Stated 
Objective or 
Big Idea 


Description of 
identifiable 
characteristics 
reflecting a beginning 
level of performance. 


Description of 
identifiable 
characteristics 
reflecting 
development and 
movement toward 
mastery of 
performance. 


Description of 
identifiable 
characteristics 
reflecting mastery 
of performance. 


Description of 
identifiable 
characteristics 
reflecting the 
highest level of 
performance. 
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SECTION 4 

Principles of 
Student Assessment 




This section examines assessment principles that must be 
considered when designing student assessment in general and 
technological literacy assessment in particular. 




SECTION 4 



Assessment 
principles are the 
basic truths, laws, 
or assumptions 
held in the use of 
assessment. 



Assessment principles are the basic truths, laws, or assumptions held in the use of 
assessment (ITEA, 2003). Effective assessment of student technological literacy: 

• Utilizes fair and equitable student assessment methods. 

(AETL Student Assessment Standard 2, Guideline C) 

• Establishes valid and reliable measurements. 

(AETL Student Assessment Standard 2, Guideline D ) 

• Reflects current research on student learning and student assessment. 

(AETL Student Assessment Standard 3, Guideline A) 

• Incorporates both formative and summative assessment. 

(AETL Student Assessment Standard 3, Guidelines B & C) 

• Enhances student learning. 

(AETL Student Assessment Standard 3, Guideline D) 

• Allows for student commonality and diversity. 

(AETL Student Assessment Standard 3, Guideline E) 

• Includes students in the assessment process. 

(AETL Student Assessment Standard 3, Guideline F) 

• Reflects current technological content. 

(AETL Student Assessment Standard 4, Guideline D) 

• Utilizes authentic assessment. 

(AETL Student Assessment Standard 4, Guideline E) 

The assessment principles contained in this section should not be considered 
an afterthought of standards-based assessment. These principles must guide 
the development, selection, and use of tools and methods. As teachers become 
comfortable with the standards-based approach presented in Section 2, considera- 
tion of the assessment principles may become “rote” (second-nature) but should 
never be forgotten. 

Utilize Fair and Equitable Student Assessment 
Methods 

Fair student assessment is not biased or discriminatory. It provides all students 
with opportunities to demonstrate their understanding. Therefore, we consider 
all student ideas, opinions, and work with equal objectivity. Equitable student 
assessment requires that we use methods and procedures most appropriate to 
our students (Suskie, 2000). For example, we might ask our first grade stu- 
dents to illustrate their understandings with a drawing rather than with a three- 
paragraph explanation. Or, it might be appropriate for us to read an assessment 
tool out loud to the class if several of our students read below grade level. We 
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recognize that even when students understand the content, they may not be 
able to demonstrate their understandings as well as their classmates, so we 
provide alternatives as appropriate. 



Additionally, an assessment tool or method is considered fair and equitable when it: 



• Assesses student learning against specific criteria. In the case of individ- 
ual assessment practices, these criteria are specific to the evidence being 
judged. 

• Makes expectations abundantly clear to students. It is necessary to take 
the time to clarify any misunderstandings students may have about our 
expectations. 

• Serves as ONE indicator of student technological literacy, not the ONLY 
indicator. Multiple factors can influence the evidence collected by a single 
tool or method, so we must consider student performance on a range of 
assignments to ensure fairness and equitability. In other words, we assess 
students often, using a variety of tools and methods. 

• Improves teaching and learning. Unless the results of assessment are com- 
pared to the teaching and learning process, neither instruction nor student 
understanding is likely to improve. Therefore, we use student assessment 
results to make sure instruction matches assessment, including teaching 
students how to perform the assessment. We do not use assessment results 
as a sorting tool to screen students either in or out of courses. 

Establish Valid and Reliable Measurements 

We systematically collect and record assessment data to ensure validity and relia- 
bility. Valid assessment measures have a premise from which assessment results 
can logically be inferred (ITEA, 2003). In other words, validity helps ensure that 
assessment matches the identified purpose. An assessment tool or method is con- 
sidered valid when it: 

• Collects evidence of student learning consistent with its identified purpose. 

• Assesses the content that students have actually learned in our classrooms. 

Reliable assessment measures can be repeated with consistent results (ITEA, 
2003). An assessment tool or method is considered reliable when it: 



"... validity 
focuses on the 
accuracy or truth 
of the information 
(data) collected in 
the assessment 
process, while 
reliability attempts 
to answer 
concerns about 
the consistency of 
the information 
(data) collected." 
(ITEA, 2003, p. 23) 



• Draws assessment data from several sources. 

• Is comparable to alternate test forms assessing the same content. 
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Reflect Current Research on Student Learning and 
Assessment 



As teachers of technology, we realize that new research on teaching, learning, and 
assessment is being conducted every day. Therefore, we make the effort to remain 
aware of current research on teaching, learning, and assessment. We adjust the 
techniques we use to assess student learning to align with current research on how 
students learn technology. We consider how students learn technology and how 
they can demonstrate their understanding. An assessment tool or method reflects 
current research on student learning and assessment when it: 

• Considers how students acquire new technological knowledge. 

• Considers how new knowledge is connected to past understandings. 

• Acknowledges how future learning can be enhanced through assessment. 

• Enables us to collect data to inform instruction and enhance student learning. 



Formative 
assessment is 
ongoing 

assessment in the 
classroom. 



Incorporate Both Formative and Summative Student 
Assessment 

Formative assessment is ongoing assessment in the classroom. We gather forma- 
tive assessment data daily “by, for example, reviewing homework, managing dis- 
cussions, asking and answering questions, listening to student conversations, and 
observing students [at work]” (NRC, 2002, p.62). The evidence we gather through 
formative assessment reveals student progress toward technological literacy. 
Formative assessment allows us to shape our teaching during the learning process 
based on what students understand (NRC, 2002). Therefore, we must deliberately 
incorporate formative assessment into our teaching. This is known as “embed- 
ding” assessment. It is ongoing. Embedding makes assessment authentic and, in 
the end, it saves valuable time for both teacher and student. An assessment tool or 
method is considered formative when it: 

• Occurs throughout instruction. 

• Determines student misconceptions. 

• Provides information on the effectiveness of instruction. 

• Reveals student progress toward technological literacy. 

• Facilitates instructional adjustment to enhance learning. 
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Summative assessment usually occurs at the end of a unit, topic, project, or prob- 
lem. Summative assessment is cumulative and indicates the level of technological 
literacy attained. “Level” suggests a degree of understanding (e.g., a range from 
novice to expert). An assessment tool or method is considered su mm ative when it: 

• Occurs at a prescribed interval. 

• Indicates the level of technological literacy attained by students. 

• Includes learning activities that build upon previous knowledge. 

Both formative and summative assessment are critical to a well-rounded student 
assessment approach. 

Enhance Student Learning 

A primary purpose of student assessment is to improve teaching and learning. 
Teachers collect data of individual student knowledge and abilities and use that 
information to improve the teaching and learning process for all students. Data 
collection is a continuous process that enables assessment to occur as an ongoing, 
integral part of instruction. We provide feedback to our students and encourage 
our students to use assessment results to modify their own learning (ITEA, 2003). 



Summative 
assessment 
occurs at 
prescribed 
intervals and 
provides 

information on the 
level of 
technological 
literacy 
attainment by 
students. 



Allow for Student Commonality and Diversity 

We must strike a balance between being objective about students while recogniz- 
ing and valuing their individuality. Every classroom will contain students with 
similar interests, cultures, abilities, socio-economic backgrounds, and special 
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When allowing 
for student 
commonality and 
diversity, 

assessors must be 
aware of specific 
laws that call for 
accommodation 
and/or 

modification to 

student 

assessment. 



needs. Students in every classroom will also differ based on those same ele- 
ments. If we want student similarities and differences to enhance teaching and 
learning, we must recognize that multiple instruments may be required to assess 
a single idea or concept (ITEA, 2003). At times, it will be necessary to adjust the 
testing environment and/or testing format to satisfy student commonality and 
diversity. In other instances, we might adjust the focus or level of rigor of an 
assessment tool or method. Such adjustments ensure that all students are provid- 
ed with equitable opportunities to demonstrate their understanding. Additionally, 
we consider the grade level and developmental level appropriateness of 
assessment tools and methods. 

An assessment tool or method allows for student commonality and diversity when it: 

• Supports student interests, cultures, abilities, socio-economic backgrounds, 
and special needs. 

• Aims content at assessing student understanding with appreciation for 
interests, cultures, abilities, socio-economic backgrounds, and special needs. 

• Provides all students with equal opportunities to successfully demonstrate 
understanding regardless of interests, cultures, abilities, socio-economic 
backgrounds, and special needs. 

• Implements a variety of assessment strategies (cognitive, psychomotor, 
affective) that address student interests, cultures, abilities, socio-economic 
backgrounds, and special needs while supporting student attainment of 
technological literacy. 

Include Students in the Assessment Process 

Few would argue against the need for students to be informed of the purpose of 
student assessment or of the expectations for their learning. It is not reasonable 
for us to expect our students to demonstrate something we have not asked them to 
demonstrate. We can go a step further by allowing our students to help us define 
the purpose and expectations of assessment. Students can assist us in identifying 
the content to be learned and appropriate assessment tools or methods for gather- 
ing assessment data. Although teachers play a significant role in directing such a 
discussion, students learn more about the assessment process when they become 
directly involved in it. 

Self-assessment activities also get students involved in the assessment process. 
Students reflect on past understandings, current understandings, and desired 
future understandings. We can require students to consider how their knowledge 
and abilities have developed and how they will try to advance their own knowl- 
edge and abilities in the future. 
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An assessment tool or method includes students in the assessment process when it: 

• Provides students with the opportunity to select specific content. 

( What do we need to know and be able to do in order to understand the 
important ideas?) 

• Enables students to select the assessment tools or methods to be implement- 
ed. (What kind of evidence will show that we really “get it,” that we under- 
stand the content? How can we demonstrate that we understand the 
important ideas?) 

• Allows students to define the expectations of student learning. (How are we 
going to be measured to see if we are learning the “right stuff? ”) 



Reflect Current Technological Content 

As educators, we understand that technology is dynamic — it changes with the 
changing needs and wants of humans. Therefore, we design student assessment 
tools and methods to be flexible and easy to modify. As advances in technology 
occur, we change our assessment tools and methods to reflect technological 
advancements. 



AETL Student 
Assessment 
Standard 4, 

Guideline D 
suggests that 
teachers must 
consistently 
accommodate for 
modification to 
student assessment. 
The meanings of the 
terms accommodate 
and modification in 
this guideline are 
not consistent with 
the meanings of 
these terms as 
traditionally used in 
education. The term 
accommodate is 
NOT used to imply 
adaptations in the 
testing environment 
and/or testing 
format. 

Accommodate in 
this guideline refers 
to the idea that 
changes will need 
to be made in the 
future. The term 
modification is NOT 
used to imply that 
adjustments are 
being made to the 
focus or level of 
rigor of assessment 
content. 

Modification in this 
guideline refers to 
the revision of 
content to ensure 
assessment tools 
and methods reflect 
current 
technologies. 
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Authentic 

assessment 

"...examines 

student 

performance on 
tasks that are 
directly related to 
what is 
considered 
worthy and 
necessary for 
developing 
technological 
literacy." (ITEA, 
2003, p. 136) 




Utilize Authentic Assessment 

Authentic assessment provides students with the opportunity to demonstrate their 
knowledge and abilities in real-world situations. An assessment tool or method is 
considered authentic when it: 

• Determines the level at which students understand content. 

• Places students in practical situations representative of the world outside of 
the classroom. 

• Incorporates cognitive, psychomotor, and affective learning elements. 

• Considers student commonality and diversity, recognizing student interests, 
cultures, abilities, socio-economic backgrounds, and special needs. Students 
understand both the test items (or expectations) and how they are expected to 
demonstrate their understanding of the content. 

• Makes clear to the students what is expected, how their learning will be 
judged, and how they will receive feedback. 



Grant Wiggins (1998) notes that for authenticity, assignments should be realistic, 
require judgment and innovation, have students actually “do” the subject, align 
with the process in the real world, possess complexity, and provide opportunities 
for performance refinement. 



52 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 




SECTION 5 

Applying Assessment Data 
and Evaluating Assessment 




This section presents some uses for assessment data 
as well as evaluative strategies for ensuring effective 

student assessment. 



SECTION 5 



We must consistently maintain data collection for accountability (AETL Student 
Assessment Standard 5, Guideline A). As we are required to use a variety of 
assessment tools and methods, we will have to gather a variety of evidence. We 
need to develop an organized way of managing a great deal of information. 
Individual educators will organize data differently according to personal styles. 
Whatever method teachers use to file or record assessment data, they need ready 
access to the information to communicate results, as appropriate, to students, par- 
ents, administrators, communities, policymakers, business and industry, and the 
general public. Teachers should be sure their system of data management is safe 
and secure. This will ensure that the rights of all students are respected and that 
assessment results are reported with confidentiality, privacy, and security. 

Making Use of Assessment Results 

Assessment data provide information that enables teachers and others to make 
decisions (see Table 3 on p. 26). All educators must pay particular attention to the 
original purpose(s) and intended audience(s) of the assessment tool or method to 
ensure that results are not interpreted out of context (ITEA, 2003). Teachers use 
assessment data to make a variety of decisions, including: 

• Improving teaching and learning. Teachers use student assessment results 
to enhance teaching and learning. Teachers verify that student assessment is 
ongoing and embedded within instruction. Teachers review evidence of 
learning throughout instruction, and instructional adjustments occur as 
appropriate to help all students attain technological literacy. Just as we 
reflect upon student assessment results to adjust instruction, students should 
reflect upon assessment results to modify their own learning (ITEA, 2003). 

• Assigning grades. Although the primary purpose of assessment is to 
improve teaching and learning, the reality is that end-of-term grades need to 
be generated. However, continuous assessment will contribute significantly 
to the grading process by providing multiple measures from which evidence 
of student learning may be judged. 

• Monitoring progress. Based on student assessment results, we can deter- 
mine whether or not students have attained a level of technological literacy 
consistent with Standards for Technological Literacy: Content for the Study 
of Technology (STL) (ITEA, 2000/2002). Such data provide information 
regarding student progress toward technological literacy. We can determine 
what progress has been made as well as what progress still needs to be made. 
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• Identifying levels of technological literacy. Student assessment results 
assist teachers in determining the level of technological literacy attained by 
students. We consider the content standards and benchmarks in STL to 
gauge whether or not our students are performing at the desired level con- 
sistent with the appropriate grade band. 

• Determining instructional effectiveness. Teachers continually refine their 
teaching practice based on student assessment results. We determine the 
reasons for the assessment results and use that information to adjust instruc- 
tion as appropriate. For example, unexpected results may indicate student 
misunderstanding or some external factor. Teachers adjust instruction to 
present content in an alternative manner and attempt to clarify misunder- 
standing. Data collected in the classroom assist teachers as we modify 
instruction to meet the needs of our students based on the evidence of stu- 
dent learning gathered by student assessment. 

• Communicating results. We should provide students with the necessary 
feedback for adjusting their own learning. Parents should be provided the 
necessary feedback for monitoring their child’s progress and making deci- 
sions about the quality of the technology program. Teachers determine 
which results to provide to administrators, communities, policymakers, 
business and industry, and the general public to inform them of student 
technological literacy. 

• Marketing and promotion. Teachers should use standards-based student 
assessment results to solicit support from stakeholders for the technology 
program by communicating the characteristics of technological literacy, its 
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importance, and the need for all students to attain it. 

• Guiding professional development decisions. Student assessment results 
can help guide professional development decisions for teachers (AETL 
Student Assessment Standard 5, Guideline B). Teachers assume the responsi- 
bility for their own continuous professional growth and seek in-service 
activities that help us maintain and enhance our professional and technolog- 
ical abilities. We base our professional development decisions on the learn- 
ing needs of our students. Some activities in which teachers might partici- 
pate include: 

1. Membership in local, state, and national professional organizations. 

2. Collaboration with other professionals in the fields of education, 
technology, and engineering. 

3. Involvement in mentoring activities with other technology teachers 
and other content area teachers. 

4. Engagement in informal education programs provided by businesses, 
industries, and museums, among others. 

• Guiding program enhancement decisions. We also use student assess- 
ment results to help guide program enhancement decisions (AETL Student 
Assessment Standard 5, Guideline Q. Teachers review student assessment 
results to remain informed about the status of the technology program. 
Teachers ensure that the purpose for collecting data was for program evalu- 
ation. Teachers and others examine results with attention to the purpose 
and intended audience to ensure that results are not interpreted out of con- 
text. Student assessment results are considered in conjunction with other 
program elements to determine overall program effectiveness. The bottom 
line is, are students attaining technological literacy ? Areas of concern are 
identified based on assessment results and may include such things as 
staffing, curricula, instruction, materials, facilities, or equipment. We ana- 
lyze student assessment results for clues and ideas regarding how the data 
can be used in program enhancement. Program enhancement decisions are 
made based on student assessment results and the need for ensuring high 
quality programs. 

Evaluating Student Assessment 

As with any process, we must evaluate the quality of student assessment. 
Appendix J is a form for teachers to use in evaluating student assessment. 
Evaluation ensures that every opportunity to link student assessment with tech- 
nological literacy standards is considered. As we reflect upon the results of stu- 
dent assessment, we must remember that we want all students to attain techno- 
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logical literacy. Judy Carr and Douglas Harris (2001, p. 59) suggest three ques- 
tions as the basis for leadership decisions in a standards-based system that ITEA 
staff adapted and expanded for use in evaluating student assessment: 1) What 
are the student results ? 2) Why are they what they arel 3) What were our expec- 
tations for teaching and learning ? 4) How do the student results compare with 
our expectations ? and 5) What strategies can be implemented to ensure consis- 
tency between what is expected and what results ? 

The results from the evaluation of student assessment can be used to either vali- 
date how well the current assessment process is actually working or to identify 
areas where improvement is desirable. As is the case in many evaluations, it is 
the parts of the system that generally need improvement or refinement, not the 
entire process. 

A Journey, Not a Destination 

At the heart of our modem technological society lies an unacknowledged 
paradox. Although the United States is increasingly defined by and 
dependent on technology and is adopting new technologies at a breath- 
taking pace, its citizens are not equipped to make well-considered deci- 
sions or to think critically about technology. As a society, we are not even 
fully aware of or conversant with the technologies we use every day. In 
short, we are not “technologically literate” (NAE & NRC, 2002, p. 1). 

Achieving technological literacy will be the result of a combined approach 
through professional development, curricula, instruction, student assessment, 
and the learning environment, coordinated to deliver the content of STL in a 
manner consistent with the standards in Advancing Excellence in 
Technological Literacy: Student Assessment, Professional Development, and 
Program Standards ( AETL)(YTEA , 2003). Realizing the vision of STL and 
AETL, that all students can and deserve to become technologically literate 
through appropriate K-12 education, will require support from educators, 
parents, and communities committed to developing a technologically literate 
citizenry. Effective student assessment enables us to document student 
progress toward technological literacy. Knowing where students are and where 
they should be — through a standards-based student assessment approach — 
enables us to develop strategies to advance student technological literacy. 
Technology is continually evolving and therefore, the standards reflecting 
technology must evolve as well. Technological literacy then, becomes a journey 
and not a destination. It is a journey to which we must all commit, if we are to 
attain a technologically literate populace. 



Developing a 
technologically 
literate citizenry 
will require 
support from 
educators, 
parents, and 
communities. 
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Listing of STL Content Standards 

From International Technology Education Association. (2000/2002). Standards for technological 
literacy: Content for the study of technology. Reston, VA: Author. 

The Nature of Technology 

Standard 1 . Students will develop an understanding of the characteristics and scope of technology. 

Standard 2. Students will develop an understanding of the core concepts of technology. 

Standard 3. Students will develop an understanding of the relationships among technologies and the 

connections between technology and other fields of study. 



Technology and Society 

Standard 4. Students will develop 
effects of technology. 

Standard 5. Students will develop 
environment. 

Standard 6. Students will develop 
use of technology. 

Standard 7. Students will develop 



an understanding of the cultural, social, economic, and political 
an understanding of the effects of technology on the 
an understanding of the role of society in the development and 
an understanding of the influence of technology on history. 



Design 

Standard 8. Students will develop an understanding of the attributes of design. 

Standard 9. Students will develop an understanding of engineering design. 

Standard 10. Students will develop an understanding of the role of troubleshooting, research and 

development, invention and innovation, and experimentation in problem solving. 



Abilities 

Standard 

Standard 

Standard 



for a Technological World 

1 1 . Students will develop the 

12. Students will develop the 
systems. 

13. Students will develop the 



abilities to apply the design process. 

abilities to use and maintain technological products and 

abilities to assess the impact of products and systems. 



The Designed World 

Standard 14. Students will develop an understanding of and be able to 
technologies. 

Standard 15. Students will develop an understanding of and be able to 
related biotechnologies. 

Standard 16. Students will develop an understanding of and be able to 
power technologies. 

Standard 17. Students will develop an understanding of and be able to 
and communication technologies. 

Standard 18. Students will develop an understanding of and be able to 
technologies. 

Standard 19. Students will develop an understanding of and be able to 
technologies. 

Standard 20. Students will develop an understanding of and be able to 
technologies. 



select and use medical 
select and use agricultural and 
select and use energy and 
select and use information 
select and use transportation 
select and use manufacturing 
select and use construction 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 



63 



APPENDIX B 




APPENDIX C 



APPENDIX C 

Listing of AETL Student Assessment Standards with Guidelines 



From International Technology Education Association. (2003). Advancing excellence in technological 
literacy: Student assessment, professional development, and program standards. Reston, VA: Author. 

Standard A-l: Assessment of student learning will be consistent with Standards for Technological 
Literacy: Content for the Study of Technology (STL). 

Guidelines for meeting Standard A- 1 require that teachers consistently 

A. Administer comprehensive planning and development across disciplines. 

B. Incorporate comprehensive planning and development across grade levels. 

C. Include cognitive learning elements for solving technological problems. 

D. Include psychomotor learning elements for applying technology. 

E. Guide student abilities to operate within the affective domain, utilizing perspective, empathy, and 
self-assessment. 

Standard A-2: Assessment of student learning will be explicitly matched to the intended purpose. 

Guidelines for meeting Standard A-2 require that teachers consistently 

A. Formulate a statement of purpose for assessment tools. 

B. Identify and consider the intended audience in designing assessment tools and reporting 
assessment data. 

C. Utilize fair and equitable student assessment methods. 

D. Establish valid and reliable measurements that are reflective of classroom experiences. 

Standard A-3: Assessment of student learning will be systematic and derived from research- 
based assessment principles. 

Guidelines for meeting Standard A-3 require that teachers consistently 

A. Remain current with research on student learning and assessment. 

B. Devise a formative assessment plan. 

C. Establish a summative assessment plan. 

D. Facilitate enhancement of student learning. 

E. Accommodate for student commonality and diversity. 

F. Include students in the assessment process. 

Standard A-4: Assessment of student learning will reflect practical contexts consistent with the 
nature of technology. 

Guidelines for meeting Standard A-4 require that teachers consistently 

A. Incorporate technological problem solving. 

B. Include variety in technological content and performance-based methods. 

C. Facilitate critical thinking and decision making. 

D. Accommodate for modification to student assessment. 

E. Utilize authentic assessment. 

Standard A-5: Assessment of student learning will incorporate data collection for accountability, 
professional development, and program enhancement. 

Guidelines for meeting Standard A-5 require that teachers consistently 

A. Maintain data collection for accountability. 

B. Use student assessment results to help guide professional development decisions. 

C. Use student assessment results to help guide program enhancement decisions. 
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Listing of AETL Professional Development Standards 

From International Technology Education Association. (2003). Advancing excellence in technological 
literacy: Student assessment, professional development, and program standards. Reston, VA: Author. 

Note: These standards are provided for reference only. All standards should be met through the 
guidelines that follow each standard in Advancing Excellence in Technological Literacy, which is 
available online at www.iteawww.org. 



Standard PD-1: Professional development will provide teachers with knowledge, abilities, and 

understanding consistent with Standards for Technological Literacy: Content for the 
Study of Technology {STL). 

Standard PD-2: Professional development will provide teachers with educational perspectives on 
students as learners of technology. 

Standard PD-3: Professional development will prepare teachers to design and evaluate technology 
curricula and programs. 

Standard PD-4: Professional development will prepare teachers to use instructional strategies that 
enhance technology teaching, student learning, and student assessment. 

Standard PD-5: Professional development will prepare teachers to design and manage learning 
environments that promote technological literacy. 

Standard PD-6: Professional development will prepare teachers to be responsible for their own 
continued professional growth. 

Standard PD-7 : Professional development providers will plan, implement, and evaluate the 
pre-service and in-service education of teachers. 
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Listing of AETL Program Standards 

From International Technology Education Association. (2003). Advancing excellence in technological 
literacy: student assessment, professional development, and program standards. Reston, VA: Author. 

Note: These standards are provided for reference only. All standards should be met through the 
guidelines that follow each standard in Advancing Excellence in Technological Literacy, which is 
available online at www.iteawww.org. 



Standard P-1: Technology program development will be consistent with Standards for Technological 
Literacy: Content for the Study of Technology (STL). 

Standard P-2: Technology program implementation will facilitate technological literacy for all 
students. 

Standard P-3: Technology program evaluation will ensure and facilitate technological literacy for all 
students. 

Standard P-4: Technology program learning environments will facilitate technological literacy for all 
students. 

Standard P-5: Technology program management will be provided by designated personnel at the 
school, school district, and statc/provincial/rcgional levels. 
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Responsibility Matrix Form 



Directions: Page 1 of this form should be used to indicate which standards in Standards for Technological Literacy 
(STL) will be addressed at each grade level of the technology program. Fill in this form using “X” to indicate maxi- 
mum coverage, “V” to indicate moderate coverage, and “O” to indicate minimal coverage. 



Responsibility Matrix Form Page 1 



STL Standards 


STL Coverage in the Technology Program 


Elementary Level Classrooms 


Technology Laboratory-Classrooms 


K 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 


STL 1. Students will develop an understanding of the 
characteristics and scope of technology. 




























STL 2. Students will develop an understanding of the core 
concepts of technology. 




























STL 3. Students will develop an understanding of the 
relationships among technologies and the connections between 
technology and other fields of study. 




























STL 4. Students will develop an understanding of the cultural, 
social, economic, and political effects of technology. 




























STL 5. Students will develop an understanding of the effects of 
technology on the environment. 




























STL 6. Students will develop an understanding of the role of 
society in the development and use of technology. 




























STL 7. Students will develop an understanding of the influence of 
technology on history. 




























STL 8. Students will develop an understanding of the attributes of 
design. 




























STL 9. Students will develop an understanding of engineering 
design. 




























STL 10. Students will develop an understanding of the role of 
troubleshooting, research and development, invention and 
innovation, and experimentation in problem solving. 




























STL 11. Students will develop the abilities to apply the design 
process. 




























STL 12. Students will develop the abilities to use and maintain 
technological products and systems. 




























STL 13. Students will develop the abilities to assess the impact of 
products and systems. 




























STL 14. Students will develop an understanding of and be able 
to select and use medical technologies. 




























STL 15. Students will develop an understanding of and be able 
to select and use agricultural and related biotechnologies. 




























STL 16. Students will develop an understanding of and be able to 
select and use energy and power technologies. 




























STL 17. Students will develop an understanding of and be able to 
select and use information and communication technologies. 




























STL 18. Students will develop an understanding of and be able to 
select and use transportation technologies. 




























STL 19. Students will develop an understanding of and be able 
to select and use manufacturing technologies. 




























STL 20. Students will develop an understanding of and be able 
to select and use construction technologies. 
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Directions: Page 2 of this form should be used to indicate which standards from other content areas will be addressed 
at each grade level of the technology program. Multiple copies of this form may be needed. Fill in this form using “X” 
to indicate maximum coverage, “V” to indicate moderate coverage, and “O” to indicate minimal coverage. 



Responsibility Matrix Form Page 2 



Other Content Area Standards 


Standards Coverage in the Technology Program 


Elementary Level Classrooms 


Technology Laboratory-Classrooms 


K 


1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


11 


12 
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Directions: Page 3 of this form should be used to indicate which standards in Standards for Technological Literacy 
(STL) will be addressed at each grade level in other content area classrooms. Multiple copies of this form may be 
needed. Fill in this form using “X” to indicate maximum coverage, “V” to indicate moderate coverage, and “O” to 
indicate minimal coverage. 



Responsibility Matrix Form Page 3 



STL Standards 


STL Coverage in Other Content Areas 






6 


7 


8 


9 


10 


11 


12 


6 


7 


8 


9 


10 


11 


12 


STL 1. Students will develop an understanding of the 
characteristics and scope of technology. 






























STL 2. Students will develop an understanding of the core 
concepts of technology. 






























STL 3. Students will develop an understanding of the 
relationships among technologies and the connections 
between technology and other fields of study. 






























STL 4. Students will develop an understanding of the 
cultural, social, economic, and political effects of technology. 






























STL 5. Students will develop an understanding of the effects 
of technology on the environment. 






























STL 6. Students will develop an understanding of the role of 
society in the development and use of technology. 






























STL 7. Students will develop an understanding of the 
influence of technology on history. 






























STL 8. Students will develop an understanding of the 
attributes of design. 






























STL 9. Students will develop an understanding of 
engineering design. 






























STL 10. Students will develop an understanding of the role 
of troubleshooting, research and development, invention and 
innovation, and experimentation in problem solving. 






























STL 1 1 . Students will develop the abilities to apply the 
design process. 






























STL 12. Students will develop the abilities to use and 
maintain technological products and systems. 






























STL 13. Students will develop the abilities to assess the 
impact of products and systems. 






























STL 14. Students will develop an understanding of and be 
able to select and use medical technologies. 






























STL 15. Students will develop an understanding of and be 
able to select and use agricultural and related 
biotechnologies. 






























STL 16. Students will develop an understanding of and be 
able to select and use energy and power technologies. 






























STL 17. Students will develop an understanding of and be 
able to select and use information and communication 
technologies. 






























STL 18. Students will develop an understanding of and be 
able to select and use transportation technologies. 






























STL 19. Students will develop an understanding of and be 
able to select and use manufacturing technologies. 






























STL 20. Students will develop an understanding of and be 
able to select and use construction technologies. 
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APPENDIX H 

Course Level 

Standards-Based Student Assessment Form 

Step 1. SELECT CONTENT STANDARDS, THEN IDENTIFY BENCHMARKS 

Identify and document the content standards that will serve as the basis for student assessment. Then select 
the benchmarks that add detail to the concepts that will be taught and assessed. Multiple copies of this page 
may be needed. 

STOP! And confirm: Review the selected content standards and benchmarks to verify that they include 
variety in technological content. 

Step 2. EXTRACT AND ORGANIZE CONTENT 

What are the specific understandings students should possess having engaged in the content of the standards and 
benchmarks? What are the big ideas students should learn to ultimately understand the standards and benchmarks? 



Content Standard 


Benchmark 








Big Idea: 


Big Idea: 


Big Idea: 



Content Standard 


Benchmark 








Big Idea: 


Big Idea: 


Big Idea: 



Content Standard 


Benchmark 








Big Idea: 


Big Idea: 


Big Idea: 
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Step 3: DEFINE ASSESSMENT CRITERIA 

What are the expectations for student Learning? Define criteria at Levels which meet, exceed, and fall below 
your expectations. 

NOTE: Defining assessment cn'ten'a at various levels requires that you first determine the number of levels that will 
be assigned. You may choose to use any of the three tables that follow or develop one of your own. 



Big Ideas 


Assessment Criteria 


Below Target 


At Target 


Above Target 



































STOP! And confirm: Review the assessment criteria defined above. Verify that they: 

• Include cognitive learning elements for solving technological problems. 

• Include psychomotor learning elements for applying technology. 

• Include affective learning elements suitable for utilizing perspective, empathy, and self assessment. 

• Incorporate technological problem solving. 

• Facilitate critical thinking and decision making. 
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Big Ideas 


Assessment Criteria 


No Response 


Inadequate 

Response 


Minimal 

Response 


Competent 

Response 


Exemplary 

Response 



















































STOP! And confirm: Review the assessment criteria defined above. Verify that they: 

• Include cognitive learning elements for solving technological problems. 

• Include psychomotor learning elements for applying technology. 

• Include affective learning elements suitable for utilizing perspective, empathy, and self assessment. 

• Incorporate technological problem solving. 

• Facilitate critical thinking and decision making. 
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Big Ideas 


Assessment Criteria 


Novice 


Apprentice 


Journeyman 


Expert 











































STOP! And confirm: Review the assessment criteria defined above. Verify that they: 

• Include cognitive learning elements for solving technological problems. 

• Include psychomotor learning elements for applying technology. 

• Include affective learning elements suitable for utilizing perspective, empathy, and self assessment. 

• Incorporate technological problem solving. 

• Facilitate critical thinking and decision making. 
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Step 4: SELECT AND USE ASSESSMENT TOOLS AND/OR METHODS 



Identify and record the tooLs and methods that will be used to assess students for this course. The tools and 
methods you seLect will depend upon how you structure assessment for the course. For example, you may or 
may not choose to develop a summative course level assessment instrument. You might choose to incorporate 
unit level assessment tools and methods, or you might choose to look at unit level assessment results holisti- 
cally. In any case, the tools and/or methods you choose should incorporate student work over time and include 
variety in both content and method. 



STOP! And confirm: Review the assessment tools and methods listed above. Verify that they indude 
variety in performance-based methods. 



Step 5: MAKE USE OF ASSESSMENT RESULTS 

Compare the assessment results with your expectations for student learning. Consider strategies to improve 
teaching and learning so that results will match expectations. 



Consider other positive ways to use the assessment results from your course. 
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APPENDIX I 

Unit Level 

Standards-Based Student Assessment Form 

Step 1: SELECT CONTENT STANDARDS, THEN IDENTIFY BENCHMARKS 

Identify and document the content standards that will serve as the basis for student assessment by 
re-examining Steps 1 and 2 at the course Level. Then seLect the benchmarks that add detail to the 
concepts that will be taught and assessed. 

STOP! And confirm: Review the selected content standards and benchmarks to verify that they include 
variety in technological content. 

Step 2. EXTRACT AND ORGANIZE CONTENT 

What are the specific understandings students should possess having engaged in the content of the standards and 
benchmarks? What are the big ideas students should learn to ultimately understand the standards and benchmarks? 



Content Standard 


Benchmark 








Big Idea: 


Big Idea: 


Big Idea: 



Content Standard 


Benchmark 








Big Idea: 


Big Idea: 


Big Idea: 



Content Standard 


Benchmark 








Big Idea: 


Big Idea: 


Big Idea: 
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Step 3: DEFINE ASSESSMENT CRITERIA 



What are the expectations for student learning? Define criteria at Levels which meet, exceed, and fall 
below your expectations. 

NOTE: Defining assessment criteria at various levels requires that you first determine the number of levels that will 
be assigned. You may choose to use any of the three tables that follow or develop one of your own. 



Big Ideas 


Assessment Criteria 


Below Target 


At Target 


Above Target 



































STOP! And confirm: Review the assessment criteria defined above. Verify that they: 

• Include cognitive learning elements for solving technological problems. 

• Include psychomotor learning elements for applying technology. 

• Include affective learning elements suitable for utilizing perspective, empathy, and self assessment. 

• Incorporate technological problem solving. 

• Facilitate critical thinking and decision making. 
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Big Ideas 


Assessment Criteria 


No Response 


Inadequate 

Response 


Minimal 

Response 


Competent 

Response 


Exemplary 

Response 



















































STOP! And confirm: Review the assessment criteria defined above. Verify that they: 

• Include cognitive learning elements for solving technological problems. 

• Include psychomotor learning elements for applying technology. 

• Include affective learning elements suitable for utilizing perspective, empathy, and self assessment. 

• Incorporate technological problem solving. 

• Facilitate critical thinking and decision making. 
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Big Ideas 


Assessment Criteria 


Novice 


Apprentice 


Journeyman 


Expert 











































STOP! And confirm: Review the assessment criteria defined above. Verify that they: 

• Include cognitive learning elements for solving technological problems. 

• Include psychomotor learning elements for applying technology. 

• Include affective learning elements suitable for utilizing perspective, empathy, and self assessment. 

• Incorporate technological problem solving. 

• Facilitate critical thinking and decision making. 
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Step 4: SELECT AND USE ASSESSMENT TOOLS AND/OR METHODS 



Identify and record the tools and methods that will be used to assess students on this unit. Selection of tools and 
methods should be based on the content being assessed, the type of evidence being gathered, and the audience 
for assessment results. 



Assessment Tool/Method 



Content Evidence Audience 



NOTE: Students should be assessed often, using a van'ety of tools and methods. STOP! And confirm: Review 
the assessment tools and methods listed above. Verify that they include variety in performance-based 
methods. 

Step 5: MAKE USE OF ASSESSMENT RESULTS 

Compare the assessment results with your expectations for student learning. Consider strategies to improve 
teaching and learning so that results will match expectations. 



Consider other positive ways to use the assessment results from your unit. 
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Evaluating Student Assessment: Have We Arrived? 









Not 




Considerations for Assessment of 


Yes 


No 


Comments 


Student Technological Literacy 


Applicable 

(N/A) 


Student assessment adheres to accepted principles. 










• Student assessment is fair and equitable. 


Yes 


No 


N/A 




• Student assessment measures are valid and reliable. 


Yes 


No 


N/A 




• Student assessment reflects current research on 


Yes 


No 


N/A 




student learning and student assessment. 










• Student assessment incorporates both formative and 


Yes 


No 


N/A 




summative assessment. 










• Student assessment considers student commonality 


Yes 


No 


N/A 




and diversity. 










• Students are included in the assessment process. 


Yes 


No 


N/A 




• Student assessment reflects current technological 


Yes 


No 


N/A 




content. 










• Authentic assessment is used. 


Yes 


No 


N/A 




Student assessment is administered across grade levels. 


Yes 


No 


N/A 




Student assessment is administered across disciplines. 


Yes 


No 


N/A 




Student assessment advances student technological literacy. 










• Students develop an understanding of the nature of 


Yes 


No 


N/A 




technology. 








• Students develop an understanding of technology 


Yes 


No 


N/A 




and society. 










• Students develop an understanding of design. 


Yes 


No 


N/A 




• Students develop abilities for a technological world. 

• Students develop an understanding of and are abLe 


Yes 


No 


N/A 




to select and use technologies of the designed 
world. 


Yes 


No 


N/A 




Assessment tools/methods clearly define the purpose for 


Yes 


No 


N/A 




student assessment. 




Assessment tools/methods identify the audience(s) to whom 


Yes 


No 


N/A 




assessment results will be provided. 




Assessment tools/methods are flexible to allow for revisions 


Yes 


No 


N/A 




based upon the changing nature of technology. 





Measuring Progress: A Guide to Assessing Students for Technological Literacy 



81 



APPENDIX J 




APPENDIX J 



Assessment tools/methods are appropriate for gathering 










evidence of student technological literacy learning. 










• 


Evidence is gathered to indicate that students 
understand the content in Standards for 
Technological Literacy. 


Yes 


No 


N/A 




• 


The evidence gathered attempts to measure student 
understanding of the big ideas that are extracted 
from the content standards. 


Yes 


No 


N/A 




• 


Evidence of cognitive, psychomotor, and affective 
learning is gathered. 


Yes 


No 


N/A 




• 


Assessment incorporates technological problem 
solving. 


Yes 


No 


N/A 




• 


Assessment facilitates critical thinking and decision 
making. 


Yes 


No 


N/A 




• 


A variety of assessment tools and methods are used. 


Yes 


No 


N/A 




Student assessment data are used. 










• 


Data are used to improve teaching and learning. 


Yes 


No 


N/A 




• 


Data are used to monitor ongoing student progress. 


Yes 


No 


N/A 




• 


Data are used to identify levels of student 
technological literacy. 


Yes 


No 


N/A 




• 


Data are used to determine instructional 
effectiveness. 


Yes 


No 


N/A 




• 


Data are communicated to the students and other 
stakeholders. 


Yes 


No 


N/A 




• 


Data are used to guide teacher professional 
development decisions. 


Yes 


No 


N/A 




• 


Data are used to guide program enhancement 
decisions. 


Yes 


No 


N/A 




Student assessment is evaluated for quality. 


Yes 


No 


N/A 
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Scoring Rubric Examples 



Example Portfolio Scoring Rubric 


Expectation 


1 

Beginning to Attain 
Expectations 


2 

Nearly Attained 
Expectations 


3 

Achieved 

Expectations 


4 

Exceeded 

Expectations 


Portfolio is 

aesthetically 

pleasing. 


Portfolio has few 
unintended mistakes 
in the way it is 
decorated and 
appears. 


Portfolio has extra 
graphic and text 
elements that accent 
contents, providing an 
interesting look. 


Portfolio has extra 
graphic and text 
elements that accent 
contents, provide an 
interesting look, and 
establish a format that 
carries over from one 
unit to the next. 


Portfolio has format 
elements that improve 
the look and interest 
of the portfolio, are 
drawn well, maintain 
the overall format, and 
are free of mistakes. 


Portfolio is 
effectively organized. 


Portfolio has broad 
categories that help 
the student to group 
design processes. 


Portfolio has 
subheadings that 
further organize the 
design process. 


Portfolio is organized 
in a way appropriate 
for the content being 
studied and has an 
appropriate amount of 
narrative explanation. 


Portfolio is organized 
by a design process 
sequence, has 
appropriate narrative 
explanation, and is 
appropriate for the 
content being studied. 


Portfolio includes 
thumbnail sketches 
of various design 
solutions to the 
problem. 


Thumbnail or rough 
sketches are evident. 


Portfolio includes 
thumbnail sketches of 
various design 
solutions to the 
problem at hand. 


Portfolio includes 
thumbnail sketches of 
various design 
solutions, and at least 
one is related to the 
final solution. 


Portfolio includes 
thumbnail sketches of 
various design 
solutions, and they 
illustrate a progression 
of idea development. 


Portfolio includes 
technical sketches. 


Technical sketches are 
evident. 


Technical sketches are 
related to subsequent 
technical drawings. 


All of those technical 
sketches necessary to 
communicate the 
solution idea are 
included and are 
scaled and 
proportional. 


Portfolio has a 
comprehensive set of 
sketches, and each 
sketch is complete and 
follows conventions 
appropriate to the 
content area. 


Portfolio includes 
orthographic 
drawings or those 
appropriate to the 
content. 


Orthographic drawings 
are evident. 


Orthographic drawings 
are related to 
subsequent pictorials 
and renderings. 


All of the orthographic 
drawings necessary to 
communicate the 
solution idea are 
included and are 
scaled and 
proportional. 


Portfolio has a 
complete set of 
orthographic drawings, 
and each drawing is 
complete and follows 
conventions 
appropriate to the 
content area. 


Portfolio includes 
pictorial drawings or 
those appropriate to 
the content. 


Pictorial drawings are 
evident. 


Pictorial drawings are 
based upon 
orthographic drawings 
and related to 
subsequent renderings. 


All of the pictorial 
drawings necessary to 
communicate the 
solution idea are 
included and are 
scaled and 
proportional. 


Portfolio has a 
complete set of 
pictorial drawings and 
each drawing is 
complete and follows 
conventions 
appropriate to the 
content area. 


Portfolio includes a 
rendering of the 
solution. 


Rendering is evident. 


Rendering obviously 
depicts the actual 
solution. 


Rendering obviously 
includes consideration 
of shape, form, color, 
and texture. 


Rendering 
demonstrates 
consideration of 
shape, form, color, 
and texture. Rendering 
looks realistic. 
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Example Modeling / Prototyping Scoring Rubric 


^^^^^Level 

Expectation 


1 

Below Target 


2 

At Target 


3 

Above Target 


Generating Ideas 


Presents a few ideas, 
primarily as sketches. 


Presents ideas with little 
variety, flexibility, or 
representation. 


Represents ideas, changes 
with new information, and 
grows as new variables are 
introduced. Uses graphic and 
physical mediums. 


Testing Ideas 


No plan for experimentation, 
tests are few, results have 
limited possibilities. 


Plan for experimentation is 
unclear, improvement is 
minimal. 


Presents a plan for 
experimentation, represents 
ideas and solicits feedback, 
and changes as variables are 
introduced. Shows 
refinement and improvement. 


Communicating 

Ideas 


The proposed solution is 
relatively unclear and 
unappealing. 


Communicates the solution 
adequately. 


Communicates the solution 
clearly and creatively using 
an appropriate medium. 



Example Design Brief Scoring Rubric 




1 


2 


3 


Expectation 


Below Target 


At Target 


Above Target 


Drawing 


Communicates the design but 
leaves much to the discretion 
of the user. 


Communicates the design 
with most of the information 
readily available. 


Communicates the design 
efficiently and effectively. 


Modeling 


Leaves in question whether 
the design will actually work. 


Suggests that the design will 
probably work. 


Makes it clear that the 
design will work. 


Documenting 


A sampling of materials 
depicts some of the work 
leading to the final solution. 


A collection of materials 
represents most of the work 
leading to the final solution. 


A well-organized portfolio 
contains all of the work 
leading to the final solution. 


Presenting 


The presentation does a 
reasonably good job of 
communicating the final 
solution. 


The presentation 
communicates the final 
solution effectively. 


The presentation creatively 
communicates the final 
solution efficiently and 
effectively. 



Example Emotional Intelligence Scoring Rubric 


Expectations 

LeveT~'~~~~~~' 


Focus 


Goals 


Modifying 

Performance 


Motivation 


Optimism 


Optimal 

Performance 


3 

Above Target 


Has a clear 
understanding of 
the task, stays 
with it, and is 
observant for 
changing 
conditions. 


Establishes 
short- and 
long-term 
goals. 


Seeks and applies 
feedback to improve 
performance. 


Self-directed 
and pursues 
goals with 
enthusiasm and 
confidence. 


Exhibits a 
positive 
attitude 
and 

influences 

others. 


Has a sense of high 
quality 

performance, and 
works toward it. 


2 

At Target 


Understands the 
task and tends to 
stay with it. 


Establishes 
only short- 
term goals. 


Accepts feedback 
with hesitation but 
does use it to 
improve 
performance. 


Pursues goals 
with 

encouragement. 


Usually has 
a positive 
attitude. 


Has a sense of high 
quality performance 
and usually works 
toward it. 


1 

Below Target 


Understands the 
task but is easily 
distracted. 


Has difficulty 

setting 

goals. 


Unwilling to use 
feedback to improve 
performance. 


Pursues tasks, 
but only with 
prodding. 


Tends to be 
pessimistic. 


Has difficulty 
pursuing high 
quality 
performance. 
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APPENDIX M 

Glossary 

The terms defined and described in this glossary apply specifically to Measuring Progress: A Guide to 
Assessing Students for Technological Literacy. These terms may have different meanings in different 
situations. 



Accommodation — 1 . Allowing for changes to be 
made. 2. Adjustment in the testing environment 
and/or testing format. 

Accountability — The quality of being held 
answerable or responsible for, which may make 
one liable to being called to account. 

Across disciplines — Inclusive of all content area 
classrooms as appropriate to develop technological 
literacy. 

Across grade levels — Inclusive of all grades 
specified in the identified levels of an institution of 
learning, such as across grades kindergarten 
through twelve for public education. 

Affective — Relating to, arising from, or influenc- 
ing feelings or emotions. 

Articulated — A planned sequence of curricula 
and course offerings from Grades K-12. The 
planned sequence may involve looking at course 
offerings across grade levels (vertical articulation) 
or the curriculum at a single grade level (horizon- 
tal articulation). 

Assessment — See student assessment. 

Assessment criteria — The expectations of stu- 
dent learning that are used for collecting informa- 
tion on student learning. They define “what a stu- 
dent should look like” and can be measured and/or 
observed. 

Assessment method — Any of the techniques 
used by teachers that enable students to demon- 
strate understanding; i.e., open-ended questioning, 
observation, etc. 

Assessment principles — The basic truths, laws, 
or assumptions held in the use of assessment. The 
assessment principles that are in current use should 
enhance student learning, provide coherency of 
programs and courses, identify expectations, 



ensure developmental appropriateness, and be 
barrier-free. 

Assessment tool — Any of the instruments com- 
pleted by students that enable them to demonstrate 
their understanding (i.e., multiple-choice test, 
design brief, etc). 

Authentic assessment — An assessment method 
that directly examines student performance on tasks 
that are directly related to what is considered worthy 
and necessary for developing technological literacy. 
Traditional assessment, by contrast, relies on indi- 
rect or stand-in tasks or questions that are more effi- 
cient and simplistic than they are helpful in deter- 
mining what students actually know and can do. 

Benchmark — In Standards for Technological 
Literacy: Content for the Study of Technology 
(ITEA, 2000/2002), it is a written statement that 
describes the specific developmental components 
by various grade bands (K-2, 3-5, 6-8, and 9-12) 
that students should know or be able to do in order 
to achieve a standard. 

Brainstorm — A method of shared problem solv- 
ing in which all members of a group spontaneous- 
ly, and in an unrestrained discussion, generate 
ideas. 

Checklist — An evaluative tool, which can take 
many forms, from a simple listing to a formal 
quarterly report of progress. 

Cognitive — 1. Having a basis in or being 
reducible to empirical, factual knowledge. 2. A 
teaching method that recognizes the close relation- 
ship between what is known and what is to be 
learned. The teaching proceeds to build on the stu- 
dent’s knowledge base by helping the student asso- 
ciate new material with something that is familiar. 

Commonality — Similarity of interests, cultures, 
abilities, socio-economic backgrounds, and/or spe- 
cial needs. 
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Concept mapping — An assessment approach 
involving the creation of a two-dimensional graph- 
ic representation that details the relationships 
among ideas. 

Content standards — 1 . The standards in Standards 
for Technological Literacy: Content for the Study of 
Technolog y that provide written statements of the 
knowledge and abilities students should possess in 
order to be technologically literate. 2. The standards 
in other content areas that specify what students 
should know and be able to do, including those in 
National Science Education Standards or Principles 
and Standards for School Mathematics. 

Context/Contextual — The circumstances in 
which an event occurs; a setting. 

Continuous — Uninterrupted in time, sequence, 
substance, or extent. 

Course — A series of units that lasts for a speci- 
fied period of time (semester, year, etc.) and is 
designed around a specified school subject. 

Criteria — Desired specifications (elements or 
features) of a product or system. 

Critical thinking — The ability to acquire infor- 
mation, analyze and evaluate it, and reach a conclu- 
sion or answer by using logic and reasoning skills. 

Curriculum/Curricula — Specification of the 
way content is delivered, including the structure, 
organization, balance, and presentation of content 
in the laboratory-classroom. 

Curriculum development — The process of cre- 
ating planned curriculum, pedagogy, instruction, 
and presentation modes. 

Debate — An open discussion “for” or “against” 
an issue or question in which two teams of three or 
four students present an argument in front of a 
classroom audience. 

Decision making — The act of examining several 
possible behaviors and selecting from them the one 
most likely to accomplish the individual’s or 
group’s intention. Cognitive processes such as rea- 
soning, planning, and judgment are involved. 



Demonstration — An assessment approach that 
involves student explanation and communication of 
their understanding of key ideas, concepts, and 
principles and their abilities of processes, tech- 
niques, and skills. 

Design — An iterative decision-making process 
that produces plans by which resources are con- 
verted into products or systems that meet human 
needs and wants or solve problems. 

Design brief — A written plan that identifies a 
problem to be solved, its criteria, and its con- 
straints. The design brief is used to encourage 
thinking of all aspects of a problem before attempt- 
ing a solution. 

Design process — A systematic problem-solving 
strategy, with criteria and constraints, used to 
develop many possible solutions to a problem or to 
satisfy human needs and wants and winnow (nar- 
row) down the possible solutions to one final 
choice. 

Developmental level appropriateness — Intended 
to match the needs of students in the areas of cog- 
nition, physical activity, emotional growth, and 
social adjustment. 

Disciplines — Specified realms of content. 

Discussion — An assessment approach that 
involves idea-sharing of subject matter between 
student and teacher or among students. Teachers 
consider student ability to verbalize content and 
make “sense” of topics, issues, or information. 

Diversity — Differences of interests, cultures, 
abilities, socio-economic backgrounds, and/or spe- 
cial needs. 

Educational (instructional) technology — 

The use of technological developments, such as 
computers, audiovisual equipment, and mass 
media, as tools to enhance and optimize the teach- 
ing and learning environment in all school sub- 
jects, including technology education. 

Educators — Those professionals involved in the 
teaching and learning process, including teachers 
and administrators. 
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Effective — Produces the desired results with 
efficiency. 

Embedded — To set or fix firmly into a statement 
or activity. 

Enduring concepts — The large, important, pro- 
found, and lasting ideas that will remain valid over 
a long period of time. 

Engineering design — The systematic and cre- 
ative application of scientific and mathematical 
principles to practical ends such as the design, 
manufacture, and operation of efficient and 
economical structures, machines, processes, 
and systems. 

Equitable — Fair, impartial, or just. 

Evaluation — The collection and processing of 
information and data to determine how well a 
design meets the requirements and to provide 
direction for improvements. 

Evidence — The information that is intended to 
demonstrate or prove a level of understanding. 

Expectations — Anticipated action that demon- 
strates understanding. 

Experimentation — 1 . The act of conducting a 
controlled test or investigation. 2. The act of trying 
out a new procedure, idea, or activity. 

Expert — Flaving specialized knowledge and/or 
ability. 

Fair — Not biased or discriminatory. 

Feedback — Using all or a portion of the informa- 
tion from the output of a system to regulate or con- 
trol the processes or inputs in order to modify the 
output. 

Forensics — The study of crime scene evidence 
for use in law, criminal investigation, and/or trial 
(e.g., gas chromatography or DNA profiling). 

Formative assessment — Ongoing assessment in 
the classroom. It provides information to students 
and teachers to improve teaching and learning. 



Grade band — A grouping of different grades in 
school (e.g., K-2, 3-5, 6-8, and 9-12). 

Grade level — A stage in the development of a 
child’s education; a grade (e.g., K, 1, 2, 3, 4, 5, 6, 

7, 8, 9, 10, 11, 12). 

Guideline — 1 . In Advancing Excellence in 
Technological Literacy: Student Assessment, 
Professional Development, and Program Standards 
(ITEA, 2003), it is a specific requirement or 
enabler that identifies what needs to be done in 
order to meet a standard. 2. A suggestion to con- 
sider. 

Holistic assessment — 1 . Integrates a collection 
of evidence to get a “big picture” or comprehen- 
sive view of what a student understands about a 
subject. 2. Emphasis of the whole, the overall, 
rather than analysis and separation into individual 
parts. 

Innovation — An improvement of an existing 
technological product, system, or method of doing 
something using both natural resources and human 
resources. 

Inputs — Something put into a system (such as 
resources) in order to achieve a result. 

Instruction — The actual teaching process that the 
teacher employs to deliver the content to all stu- 
dents. 

Intelligence — The capacity to acquire knowledge 
and the skilled use of reason; the ability to com- 
prehend. 

Interview — A form of discussion that includes a 
planned sequence of questions, similar to a job 
interview. Students are not given information, as 
the objective is to collect data on student knowl- 
edge and abilities at a certain point in time. 

Invention — A new product, system, or process 
that has never existed before, created by study and 
experimentation. 

Journal — A record of understandings, reflec- 
tions, and/or opinions written as periodic entries 
(daily, weekly). 
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Knowledge — 1 . The body of truth, information, 
and principles acquired by mankind. 2. Interpreted 
information that can be used. 

Laboratory-classroom — The environment in 
which student learning related to the study of tech- 
nology takes place. At the elementary school level, 
this environment will likely be a regular classroom. 
At the middle and high school levels, a separate 
laboratory-classroom with areas for hands-on activ- 
ities as well as group instruction, could constitute 
the environment. 

Large-scale assessment — An assessment tool or 
method that involves a large number of students, 
such as across a state/province/region or nation. 

Learning activities — Experiences provided to 
students that enable them to gain understandings. 

Learning environment — Formal or informal 
location where learning takes place that consists of 
space, equipment, resources (including supplies 
and materials), and safety and health requirements. 

Level of understanding — A degree of knowl- 
edge and/or ability that indicates understanding. 

Log — A record of understandings, reflections, 
and/or opinions written as periodic entries (daily, 
weekly). 

Mathematics — The study of abstract patterns and 
relationships that results in an exact language used 
to communicate about them. 

Measurements — Collecting data in a quantifi- 
able manner. 

Modeling — The act of creating a model, such as 
a visual, mathematical, or three-dimensional repre- 
sentation in detail of an object or design, that is 
used to test ideas, make changes to a design, 
and/or to learn more about what would happen to a 
similar, real object. 

Modification — 1 . Changing to ensure accuracy. 2. 
Adjustment in focus or level of rigor. 

Novice — A beginner, one with little or minimal 
expertise. 



Observation — The act or practice of noting and 
recording facts and events. 

Open-ended questioning — An assessment 
approach in which the teacher guides the direction, 
understanding, and application of the information 
being taught through the use of questions (and also 
attempts to identify student misconceptions) and 
uses that information to adjust instruction. 

Outputs — The results of the operation of any 
system. 

Paper-and-pencil tests — An assessment method 
that involves the use of questions that are typically 
answered in a timed setting using paper and pencil. 

Peer assessment — An assessment method that 
involves the use of feedback from one student to 
another student, both students being of similar 
standing (grade level). 

Performance — A demonstration of student- 
applied knowledge and abilities, usually by pre- 
senting students with a task or project and then 
observing, interviewing, and evaluating their solu- 
tions and products in order to assess what they 
actually know and are able to do. 

Plan/Planning — A set of steps, procedures, or 
programs worked out beforehand in order to 
accomplish an objective or goal. 

Policymakers — 1 . Those representatives inside 
the educational, public, and governmental systems 
who are responsible for public education at school, 
school district, state/provincial/regional, and 
national/federal levels. 2. Those individuals, busi- 
nesses, or groups outside the public educational 
system who influence educational policy. This may 
include parents, clubs, organizations, 
businesses/industries, political activists, and any 
number of other citizens or groups of citizens who, 
while not directly and legally responsible for creat- 
ing educational policy, nevertheless influence edu- 
cational policy. 

Portfolio — An assessment approach that involves 
the formal or informal, systematic, and organized 
collection of student work that includes results of 
research, successful and less successful ideas, 



92 



Measuring Progress: A Guide to Assessing Students for Technological Literacy 




notes on procedures, and data collected. A portfo- 
lio may be in many forms, from photographs 
depicting student growth and understanding to a 
specialized electronic journal showing work com- 
pleted over a period of time. 

Practical contexts — Everyday environments in 
which an event is likely to take place. 

Presentation — An assessment approach that 
involves the performance or delivery of informa- 
tion. 

Problem solving — The process of understanding 
a problem, devising a plan, carrying out the plan, 
and evaluating the plan in order to solve a problem 
or meet a need or want. 

Process — 1 . Human activities used to create, 
invent, design, transform, produce, control, main- 
tain, and use products or systems. 2. A systematic 
sequence of actions that combines resources to 
produce an output. 

Product — A tangible artifact produced by means 
of either human or mechanical work, or by biologi- 
cal or chemical processes. 

Professional development — A continuous 
process of lifelong learning and growth that begins 
early in life, continues through the undergraduate, 
pre-service experience, and extends through the in- 
service years. 

Program — Everything that affects student learn- 
ing, including content, professional development, 
curricula, instruction, student assessment, and the 
learning environment, implemented across grade 
levels. 

Project — A teaching or assessment method used 
to enable students to apply their knowledge and 
abilities. These may take many forms and are lim- 
ited by time, resources, and imagination. 

Prototyping — The act of creating a prototype, 
such as an original type, form, or instance, that 
serves as a full-scale working model on which later 
stages are based or judged. 



Psychomotor — 1 . Physical behavior that has a 
basis in mental processes. 2. A teaching method 
that involves both mental processes and physical 
movement. 

Reliable/Reliability — Capable of being relied 
upon; dependable; may be repeated with consistent 
results. 

Research — Systematic, scientific, documented 
study. 

Research and development (R & D) — The 

practical application of scientific and engineering 
knowledge for discovering new knowledge about 
products, processes, and services and then apply- 
ing that knowledge to create new and improved 
products, processes, and services that fill market 
needs. 

Resource — The things needed to get a job done. 
In a technological system, the basic technological 
resources are: energy, capital, information, 
machines and tools, materials, people, and time. 

Rubric — An assessment or evaluative device 
based on the identified criteria taken from the con- 
tent standards. Points or words are assigned to each 
phrase or level of accomplishment. This method 
gives feedback to the students about their work in 
key categories, and it can be used to communicate 
student performance to parents and administrators. 

Science — Understanding the natural world. 

Society — A community, nation, or broad grouping 
of people having common traditions, institutions, 
and collective activities and interests. 

Stakeholders — Individuals or entities who have 
an interest in the success of a specific venture or 
program. Stakeholders in technology education 
may include teachers, administrators, school lead- 
ers, professional development providers, business 
and industry leaders, engineers, scientists, technol- 
ogists, and others. 

Standard — A written statement or statements 
about what is valued that can be used for making a 
judgment of quality. 
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Standards-based — Educational standards pro- 
vide the content basis on which student learning is 
built. Everything that affects student learning is 
planned to support students as they attain 
standards. 

Standards-based reform — An educational move- 
ment that supports maintaining high academic 
expectations, or standards, for all students that holds 
schools, teachers, and students accountable for stu- 
dent learning and achievement. 

Standards-reflected — A connection is made to 
educational standards, but standards do not neces- 
sarily provide the basis for student learning. 
Teaching and assessment of standards is “hit 
or miss.” 

Student assessment — A systematic, multi-step 
process of collecting evidence on student learning, 
understanding, and abilities and using that informa- 
tion to inform instruction and provide feedback to 
the learner, thereby enhancing learning. 

Study of technology — Any formal or informal 
education about human innovation, change, or 
modification of the natural environment. 

Summative assessment — Cumulative assessment 
that usually occurs at the end of a unit, topic, proj- 
ect, or problem. It identifies what students have 
learned and also judges student performance 
against previously identified standards. Summative 
assessment is most often thought of as “final 
exams,” but it may also be a portfolio of 
student work. 

System — A group of interacting, interrelated, or 
interdependent elements or parts that function 
together as a whole to accomplish a goal. 

Systematic — Occurring on a regular basis; hav- 
ing a plan or order. 

Teaching — The conscious effort to bring about 
learning in a manner that is clearly understood by 
the learner and likely to be successful. 

Technological literacy — The ability to use, man- 
age, understand, and assess technology. 



Technological literacy standards — The standards 
in Standards for Technological Literacy: Content for 
the Study of Technology’ and Advancing Excellence 
in Technological Literacy: Student Assessment, 
Professional Development, and Program Standards 
that identify the content and provide criteria for the 
implementation of that content for developing tech- 
nological literacy. 

Technology — The innovation, change, or modifi- 
cation of the natural environment to satisfy per- 
ceived human needs and wants. 

Technology education — A school subject specif- 
ically designed to help students develop technolog- 
ical literacy. 

Technology program — Everything that affects 
student attainment of technological literacy, includ- 
ing content, professional development, curricula, 
instruction, student assessment, and the learning 
environment, implemented across grade levels, as a 
core subject of inherent value. 

Test — l.A method for collecting data. 2. A pro- 
cedure for critical evaluation. 

Troubleshooting — Locating and finding the 
cause of problems related to technological products 
or systems. 

Understanding — A synthesis of knowledge and 
abilities that involves sophisticated insights and is 
reflected through performance in various contexts. 

Unit — An organized series of learning activities, 
lectures, projects, and other teaching strategies that 
focuses on a specific topic related to the curricu- 
lum as a whole. 

Valid/Validity — Having or containing premises 
from which the conclusion may logically be 
derived, correctly inferred, or deduced. 

Vignette — A brief description or verbal snapshot 
of how a standard or group of standards may be 
implemented in the laboratory-classroom. 
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